feat(bitable): formula relations v2 — U1-U8 + review fixes (#26)
PR #26: bitable v2 module - Formula engine: 10→64 functions - Cross-table formula syntax - Relation CRUD (1:1/1:N/N:N + bidirectional + self-ref) - Lookup & Rollup field types with recalc worker - Cross-table dependency graph - Automation engine (3 triggers + 5 actions + retries + Webhook) - 11 REST API endpoints + 17 BitableTool actions Bug fixes (review + AE1): - SSRF guard for outbound webhooks - Bidirectional association reverse field cleanup - Target table ownership checks - Typed AutomationUpdate models - Decimal handling for JSONB numbers - merge_record_values (preserve fields on link update) - enqueue_recalc ON CONFLICT DO UPDATE (reset done→pending) - _trigger_cross_table_recalc on record update Tests: 267 unit passed, AE1 e2e 3/3 passed, ruff clean
This commit is contained in:
commit
8063f4efe0
|
|
@ -0,0 +1,762 @@
|
|||
---
|
||||
title: Bitable 公式与关联联动 v2 - Plan
|
||||
type: feat
|
||||
date: 2026-07-06
|
||||
topic: bitable-formula-relations-v2
|
||||
artifact_contract: ce-unified-plan/v1
|
||||
artifact_readiness: implementation-ready
|
||||
product_contract_source: ce-brainstorm
|
||||
execution: code
|
||||
---
|
||||
|
||||
# Bitable 公式与关联联动 v2 - Plan
|
||||
|
||||
## Goal Capsule
|
||||
|
||||
**Objective:** 把 bitable 从 "agent companion" 推进到"公式+关联联动的数据建模层"，向飞书/Notion/Baserow 等成熟产品看齐。v2 聚焦公式引擎扩展、表关联（含多对多）、跨表公式与依赖追踪、轻量自动化，并以 agent 生成公式/自动化作为差异化亮点。
|
||||
|
||||
**Product authority:** 产品负责人（用户）已确认范围；技术实现细节由 ce-plan 决定。
|
||||
|
||||
**Open blockers:** 无。三个 call-out（多对多纳入 v2、agent 差异化亮点、存储模型扩展）均已确认。
|
||||
|
||||
## Product Contract
|
||||
|
||||
**Product Contract unchanged** — R1-R20 / F1-F4 / AE1-AE5 来自 ce-brainstorm，未做产品范围变更。Planning Contract 在下方追加。
|
||||
|
||||
### Summary
|
||||
|
||||
v2 把 bitable 的公式引擎从 10 函数扩展到 40+（含跨表引用），同步引入单向/双向/多对多表关联 + Rollup 聚合 + 存储模型扩展，轻量自动化（Webhook + 触发器 + 动作）打底，并以 agent 生成公式/自动化作为差异化亮点。协作与视图扩展留给 v3+。
|
||||
|
||||
### Problem Frame
|
||||
|
||||
当前 bitable 定位为 AgentKit 的 "companion service"，已有 9 字段类型、5 视图枚举（仅 grid 实现）、10 个公式函数、3 个数据导入源、无自动化、无表关联、无实时协作。
|
||||
|
||||
竞品对比（2026 年数据）显示巨大差距：
|
||||
|
||||
| 维度 | 当前 | 飞书 | Notion | Baserow | NocoDB |
|
||||
|---|---|---|---|---|---|
|
||||
| 公式函数 | 10 | 100+ | 80+ | 100+ | Excel 兼容 |
|
||||
| 自动化 | 无 | 工作流+AI Agent | 4 触发器+4 动作 | Workflow Builder | Workflows+Scripts |
|
||||
| 表关联 | lookup 仅显示 | 单/双向/跨表公式 | 单/双向/自引用/Rollup | Link/Lookup/Rollup | LTAR V2 |
|
||||
| 字段类型 | 9 | 26 | 21-23 | 25+ | 40+ |
|
||||
|
||||
目标是对标补齐，做完整的多人协作多维表格产品。v2 聚焦单用户深度（公式+关联），协作与视图扩展留给后续版本。
|
||||
|
||||
### Key Decisions
|
||||
|
||||
**公式+关联同步做（方案 B）。** 跨表公式依赖关联基础设施，分开做会导致公式引擎返工。同步推进避免架构级重构。
|
||||
|
||||
**多对多关联纳入 v2。** 多对多需扩展存储模型（junction table 或等价方案），增加架构工作量。尽早实施避免 v3 返工——关联是数据建模的灵魂，缺多对多则复杂业务场景无法支撑。
|
||||
|
||||
**自动化轻量化。** v2 仅做 Webhook + 3 触发器 + 5 动作，不做画布式工作流编排（if/else/switch/循环）和 AI Agent 节点。优先把数据建模层做扎实，自动化深度留给 v3+。
|
||||
|
||||
**Agent 生成公式/自动化作为差异化亮点。** 飞书 AI Agent 节点很强，但我们是 agent-native 架构——agent 是"一等公民"。agent 通过自然语言为用户生成复杂公式、描述目标生成自动化规则，这是竞品没有的独特价值。
|
||||
|
||||
**视图类型全部推迟到 v3。** 牺牲短期"完整度感知"换取公式+关联的深度。kanban/gallery/gantt/form/calendar 等视图留给 v3。
|
||||
|
||||
### Actors
|
||||
|
||||
- **用户（User）**：创建/编辑表、字段、记录、公式、关联、自动化规则；消费数据
|
||||
- **Agent**：通过 BitableTool REST API 读写数据；生成公式/自动化规则（差异化能力）；upsert agent-owned 字段
|
||||
- **系统（Recalc Worker）**：异步重算公式字段（单表+跨表）；处理自动化触发器
|
||||
|
||||
### Requirements
|
||||
|
||||
#### 公式引擎扩展
|
||||
|
||||
R1. 公式函数从 10 扩展到 40+，覆盖日期（15+，如 DATE/DATEDIF/NETWORKDAYS/WEEKDAY/EOMONTH）、文本（10+，如 FIND/MID/REPLACE/SUBSTITUTE/UPPER/LOWER）、逻辑（10+，如 IFS/SWITCH/IFERROR/ISBLANK/AND/OR/NOT）、条件聚合（5+，如 SUMIF/COUNTIF/AVERAGEIF）、查找引用（5+，如 LOOKUP/FILTER/MATCH）五大类别。
|
||||
|
||||
R2. 支持跨表公式引用——公式可引用关联表的字段（如 `[关联表].[字段]` 整列引用或 `{关联字段}.{目标字段}` 行级引用），依赖关联基础设施（见 R5）。
|
||||
|
||||
R3. 公式引擎保持 AST 白名单安全机制（KTD7），新增函数需注册到白名单，禁止 `eval()`/`exec()`。
|
||||
|
||||
R4. 公式嵌套深度上限设为 15 层（对齐 Notion），超过则报错提示。
|
||||
|
||||
#### 表关联与数据模型
|
||||
|
||||
R5. 支持三种关联类型：单向关联（A→B，B 无反向字段）、双向关联（A↔B，自动创建反向字段）、多对多关联（通过 junction table 或等价机制）。
|
||||
|
||||
R6. 关联字段显示被关联记录的主键值；可通过 Lookup 字段拉取关联记录的其他字段；可通过 Rollup 字段对关联记录聚合（sum/count/avg/min/max/unique）。
|
||||
|
||||
R7. 支持自引用关联（关联"当前表"），用于父子层级/依赖关系建模。
|
||||
|
||||
R8. 关联关系删除时，依赖该关联的 Lookup/Rollup/跨表公式字段需级联处理（报错提示或级联删除，由 force 参数控制，对齐现有字段删除 guard 模式）。
|
||||
|
||||
R9. 存储模型扩展以支持关联——JSONB 记录存储关联引用（记录 ID 列表），junction table 存储多对多关系。具体 schema 由 ce-plan 决定。
|
||||
|
||||
#### 跨表公式与依赖追踪
|
||||
|
||||
R10. Recalc worker 从单表内重算扩展到跨表依赖追踪——当表 A 的记录变更时，表 B 中引用该记录的公式/Rollup 字段需触发重算。
|
||||
|
||||
R11. 跨表依赖图构建——维护字段级依赖关系（哪些表的哪些字段依赖当前记录），支持拓扑排序避免循环。
|
||||
|
||||
R12. 跨表重算异步执行，不阻塞原始记录写入；通过现有 recalc queue 机制（FOR UPDATE SKIP LOCKED）扩展。
|
||||
|
||||
#### 轻量自动化
|
||||
|
||||
R13. 支持 3 种触发器：记录新增、记录修改（可指定字段）、记录删除。
|
||||
|
||||
R14. 支持 5 种动作：新增记录、修改记录、发送 HTTP 请求（Webhook 出站）、发送邮件、发送消息（集成现有消息总线）。
|
||||
|
||||
R15. 支持 Webhook 入站——外部系统 POST 到指定端点触发自动化规则。
|
||||
|
||||
R16. 自动化规则配置存储在 bitable schema 中（新增 automation 表或扩展现有 meta），具体 schema 由 ce-plan 决定。
|
||||
|
||||
R17. 自动化规则执行异步，失败重试 3 次后标记错误状态；执行日志可查。
|
||||
|
||||
#### Agent 集成与差异化
|
||||
|
||||
R18. BitableTool 扩展新 action：`generate_formula`（自然语言描述 → 生成可用公式）、`generate_automation`（描述目标 → 生成自动化规则）、`create_relation`（创建关联字段）、`create_rollup`（创建 Rollup 字段）。
|
||||
|
||||
R19. Agent 生成的公式/自动化规则需经用户确认后落地（或直接落地但可撤销，由 ce-plan 决定交互模式）。
|
||||
|
||||
R20. Agent 生成公式时，公式引擎的 validate-formula 端点（已存在）复用，确保生成结果语法正确。
|
||||
|
||||
### Key Flows
|
||||
|
||||
**F1. 跨表公式重算流程**
|
||||
- **触发：** 用户或 agent 修改表 A 记录的字段值
|
||||
- **步骤：** 1) 识别表 A 中依赖该字段的公式字段，加入 recalc queue；2) 查跨表依赖图，找出表 B 中引用表 A 该记录的公式/Rollup 字段；3) 跨表依赖任务加入 recalc queue（标记跨表）；4) Recalc worker 按拓扑排序处理；5) 写回重算结果
|
||||
- **Covers R10, R11, R12.**
|
||||
|
||||
**F2. 多对多关联创建流程**
|
||||
- **触发：** 用户或 agent 在表 A 创建多对多关联字段，指向表 B
|
||||
- **步骤：** 1) 创建 junction table（或等价结构）存储 A-B 关联对；2) 在表 A 创建关联字段（显示 B 记录列表）；3) 在表 B 创建反向关联字段（双向，显示 A 记录列表）；4) 更新跨表依赖图
|
||||
- **Covers R5, R9.**
|
||||
|
||||
**F3. Agent 生成公式流程**
|
||||
- **触发：** 用户通过 agent 描述需求（如"计算订单金额 = 单价 × 数量"）
|
||||
- **步骤：** 1) Agent 调用 BitableTool `generate_formula` action；2) Agent LLM 生成公式表达式；3) 调用 validate-formula 端点验证语法；4) 验证通过 → 创建公式字段；5) Recalc worker 触发重算
|
||||
- **Covers R18, R19, R20.**
|
||||
|
||||
**F4. 自动化规则执行流程**
|
||||
- **触发：** 记录变更满足自动化规则的触发条件
|
||||
- **步骤：** 1) 触发器检测变更；2) 匹配自动化规则；3) 异步执行动作（新增记录/HTTP/邮件/消息）；4) 失败重试 3 次；5) 写执行日志
|
||||
- **Covers R13, R14, R15, R17.**
|
||||
|
||||
### Scope Boundaries
|
||||
|
||||
**Deferred for later (v3+):**
|
||||
- 新视图类型：kanban、gallery、gantt、form、calendar、timeline
|
||||
- 仪表盘/Dashboard、Chart 视图、15+ 图表组件
|
||||
- 画布式工作流编排：if/else/switch/循环节点、AI Agent 节点
|
||||
- 工作流画布编辑器 UI
|
||||
- Formula 2.0 高级特性：`let`/`lets` 局部变量、`map`/`filter`/`find` 列表操作、富数据输出（返回 page/date/people/list）
|
||||
|
||||
**Outside this product's identity (v4+ 协作阶段):**
|
||||
- 实时协作（OT/CRDT 算法，多人并发编辑）
|
||||
- 分享/协作权限模型（行级/列级/记录级 RBAC）
|
||||
- 评论与 @ 提及
|
||||
- 视图级权限（restricted view）
|
||||
|
||||
**Explicitly excluded:**
|
||||
- 跨多维表格文件关联（飞书也不支持，需通过同步数据）
|
||||
- 单元格内插入公式（仅公式字段支持公式，对齐飞书/Notion 模式）
|
||||
|
||||
### Deferred to Follow-Up Work
|
||||
|
||||
- 前端公式编辑器智能补全（函数名/字段名 autocomplete）— 后端 API 已支持，前端深度优化留给后续打磨
|
||||
- 自动化规则的可视化编排 UI — v2 仅做表单式配置，画布式 UI 推迟
|
||||
- 跨表公式的 EXPLAIN 依赖图可视化 — 后端依赖图已构建，可视化留给后续
|
||||
|
||||
### Acceptance Examples
|
||||
|
||||
**AE1. 跨表公式重算（Covers R2, R10, R11）**
|
||||
- **Given:** 表 Orders 有字段 `amount`，表 Customers 有 Rollup 字段 `total_amount` = SUM(Orders.amount) where Orders.customer_id = this.id
|
||||
- **When:** 用户修改 Orders 表某记录的 `amount` 值
|
||||
- **Then:** Customers 表对应客户的 `total_amount` 异步重算并更新
|
||||
|
||||
**AE2. 多对多关联（Covers R5, R6, R9）**
|
||||
- **Given:** 表 Students 和表 Courses
|
||||
- **When:** 用户在 Students 表创建多对多关联字段指向 Courses
|
||||
- **Then:** 1) junction table 创建；2) Students 显示关联的 Courses 列表；3) Courses 显示反向关联的 Students 列表；4) 在 Students 可创建 Lookup 拉取 Course 的 `credit` 字段；5) 可创建 Rollup 统计 Student 选课的总学分
|
||||
|
||||
**AE3. 公式函数扩展（Covers R1, R3）**
|
||||
- **Given:** 表有 `start_date`（date）、`end_date`（date）字段
|
||||
- **When:** 用户创建公式字段 `=NETWORKDAYS({start_date}, {end_date})`
|
||||
- **Then:** 公式验证通过，重算出工作日天数
|
||||
|
||||
**AE4. 自动化触发与执行（Covers R13, R14, R17）**
|
||||
- **Given:** 表 Tasks 有自动化规则：当 `status` 修改为"已完成"时，调用 Webhook
|
||||
- **When:** 用户将某记录 `status` 改为"已完成"
|
||||
- **Then:** 1) 触发器检测到变更；2) 异步执行 Webhook 动作；3) 执行日志记录结果；4) 若 Webhook 失败，重试 3 次后标记错误
|
||||
|
||||
**AE5. Agent 生成公式（Covers R18, R19, R20）**
|
||||
- **Given:** 表有 `price`、`quantity` 字段
|
||||
- **When:** 用户通过 agent 说"帮我算总价"
|
||||
- **Then:** 1) Agent 调用 `generate_formula` action；2) 生成公式 `={price} * {quantity}`；3) validate-formula 验证通过；4) 创建公式字段 `total_price`；5) 重算所有记录
|
||||
|
||||
### Dependencies / Assumptions
|
||||
|
||||
- **假设：** 现有 PostgreSQL JSONB 存储模型可扩展支持关联引用（记录 ID 列表嵌入 JSONB 或 junction table）
|
||||
- **假设：** 现有 recalc worker（FOR UPDATE SKIP LOCKED）可扩展到跨表依赖追踪，无需替换为消息队列
|
||||
- **依赖：** 现有 BitableTool REST API 边界（KTD5）保持不变，新 action 通过 HTTP 调用
|
||||
- **依赖：** 现有公式引擎 AST 白名单（KTD7）扩展，不引入 `eval()`/`exec()`
|
||||
|
||||
### Sources / Research
|
||||
|
||||
**竞品调研（2026 年数据）：**
|
||||
- 飞书多维表格：6 视图 + 仪表盘(15+)、26 字段类型、100+ 公式函数、工作流+AI Agent 节点、单/双向关联/跨表公式
|
||||
- Notion：11 视图、21-23 字段类型、80+ 公式（Formulas 2.0，let/lambda/map/filter）、4 触发器+4 动作自动化、单/双向/自引用/Rollup
|
||||
- Baserow：6 视图、25+ 字段类型、100+ Excel 风格公式、Workflow Builder、Link/Lookup/Rollup/Count
|
||||
- NocoDB：9 视图（含 Gantt/Timeline/Map）、40+ 字段类型、LTAR V2 统一 junction-table
|
||||
- APITable：7 视图（含 Mindmap）、24 字段类型、OT 实时协作（ot-json0+Yjs）、Magic Link 双向关联
|
||||
- Twenty：3 视图、17+ 字段类型、CRM-as-code SDK、原生 MCP Server
|
||||
|
||||
**当前实现 grounding：**
|
||||
- Dossier 路径：`/tmp/compound-engineering/ce-brainstorm/bitable-1783306851/grounding.md`
|
||||
- 关键文件：`src/agentkit/bitable/models.py`（FieldType/ViewType/FieldOwner 枚举）、`formula/engine.py`（DAG+拓扑排序）、`recalc_worker.py`（异步重算）、`server/routes/bitable.py`（API 端点）、`tools/bitable_tool.py`（agent REST 边界）
|
||||
|
||||
---
|
||||
|
||||
## Planning Contract
|
||||
|
||||
### High-Level Technical Design
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "Storage Layer (Schema V3)"
|
||||
DB[(bitable DB)]
|
||||
JT[junction tables<br/>多对多]
|
||||
AUTO[bitable_automations<br/>+ bitable_automation_logs]
|
||||
DEP[bitable_cross_table_deps<br/>依赖图]
|
||||
end
|
||||
|
||||
subgraph "Formula Engine"
|
||||
FE[FormulaEngine<br/>DAG + topo sort]
|
||||
FUNC[40+ functions<br/>date/text/logical/agg/lookup]
|
||||
XCTBL[Cross-table ref resolver<br/>relation_field.target_field]
|
||||
end
|
||||
|
||||
subgraph "Relation Layer"
|
||||
REL[RelationService<br/>single/bi/multi/self-ref]
|
||||
LK[Lookup evaluator]
|
||||
RU[Rollup evaluator<br/>sum/count/avg/min/max/unique]
|
||||
end
|
||||
|
||||
subgraph "Recalc Pipeline"
|
||||
RW[RecalcWorker<br/>FOR UPDATE SKIP LOCKED]
|
||||
DG[DependencyGraph<br/>field-level cross-table]
|
||||
Q[recalc_queue<br/>+ cross_table flag]
|
||||
end
|
||||
|
||||
subgraph "Automation Engine"
|
||||
TRIG[Trigger detector<br/>create/update/delete]
|
||||
ACT[Action executor<br/>5 actions + retry]
|
||||
WH[Webhook in/out]
|
||||
LOG[Execution logs]
|
||||
end
|
||||
|
||||
subgraph "Agent Surface"
|
||||
BT[BitableTool<br/>REST boundary KTD5]
|
||||
GF[generate_formula]
|
||||
GA[generate_automation]
|
||||
CR[create_relation]
|
||||
CU[create_rollup]
|
||||
end
|
||||
|
||||
DB --> JT
|
||||
DB --> AUTO
|
||||
DB --> DEP
|
||||
FE --> FUNC
|
||||
FE --> XCTBL
|
||||
REL --> LK
|
||||
REL --> RU
|
||||
REL --> JT
|
||||
RW --> DG
|
||||
RW --> Q
|
||||
RW --> FE
|
||||
TRIG --> ACT
|
||||
ACT --> WH
|
||||
ACT --> LOG
|
||||
BT --> GF
|
||||
BT --> GA
|
||||
BT --> CR
|
||||
BT --> CU
|
||||
GF --> FE
|
||||
CR --> REL
|
||||
CU --> RU
|
||||
XCTBL --> DG
|
||||
RU --> DG
|
||||
LK --> DG
|
||||
```
|
||||
|
||||
**关键数据流：**
|
||||
|
||||
1. **写入路径**：用户/agent 修改记录 → service 层识别受影响字段 → 查跨表依赖图 → enqueue 本表 + 跨表 recalc 任务 → worker 异步处理
|
||||
2. **重算路径**：worker claim 任务 → 构建引擎 → 解析跨表引用（通过 relation field → target field）→ 拉取关联记录值 → evaluate → 写回
|
||||
3. **自动化路径**：record mutation hook → trigger matcher → async action executor → retry 3x → log
|
||||
|
||||
### Key Technical Decisions
|
||||
|
||||
**KTD-1. Junction table 用独立表，不用 JSONB 数组。**
|
||||
多对多关联用独立 `bitable_relation_links` 表存储 `(relation_field_id, source_record_id, target_record_id)` 三元组。理由：1) 支持索引和反向查询；2) JSONB 数组在记录多时性能退化；3) 对齐 NocoDB LTAR V2 / Baserow Link 表模式。一对一/一对多关联仍在记录 JSONB 中存 `["rec_id_1", "rec_id_2"]` 数组（轻量，无独立表开销）。
|
||||
|
||||
**KTD-2. 跨表公式语法用 `{relation_field}.{target_field}` 行级引用 + `[relation_field].[target_field]` 整列引用。**
|
||||
对齐 Notion 的 `relation.property` 语法。行级引用返回当前记录关联的目标记录的该字段值（标量或列表）；整列引用返回关联表所有记录该字段的列表（用于 SUMIF 等）。parser 扩展 `_FIELD_REF_RE` 支持 `.` 分隔的二级路径。
|
||||
|
||||
**KTD-3. 跨表依赖图用物化表 `bitable_cross_table_deps`，不用运行时计算。**
|
||||
字段创建/更新时，service 层解析公式 + relation 结构，写入 `(source_table_id, source_field_id, target_table_id, target_field_id, dep_type)` 记录。recalc worker 通过该表 O(1) 查找跨表依赖，避免每次重算都解析 AST。字段变更时同步更新该表（service 层 hook）。
|
||||
|
||||
**KTD-4. 自动化执行用独立 asyncio task，不进 recalc queue。**
|
||||
自动化触发器在 service 层 record mutation hook 中同步匹配，匹配到的规则创建独立 asyncio task 执行动作。理由：1) 自动化动作（HTTP/邮件）延迟特性与公式重算不同；2) 失败重试逻辑独立；3) 避免污染 recalc queue 的 FOR UPDATE SKIP LOCKED 语义。重试用 exponential backoff（1s/2s/4s），3 次失败后标记 error。
|
||||
|
||||
**KTD-5. Agent 生成公式/自动化用"直接落地 + 可撤销"模式，不做确认弹窗。**
|
||||
agent 生成结果直接创建字段/规则，但在响应中返回 `created_id` 和 `undo_token`。用户可在 30 秒内通过 `undo` action 撤销。理由：1) agent-native 架构强调低延迟；2) 确认弹窗打断 agent 流；3) 撤销比确认更符合人类直觉。`undo_token` 存 Redis（5 分钟 TTL）。
|
||||
|
||||
**KTD-6. 公式函数扩展按类别分模块，不堆在一个文件。**
|
||||
当前 `functions.py` 已有 10 个函数。扩展到 40+ 后单文件过大。按类别拆分：`functions/datetime.py`、`functions/text.py`、`functions/logical.py`、`functions/aggregate.py`（含条件聚合）、`functions/lookup.py`。`functions/__init__.py` 聚合注册到 `FUNCTION_REGISTRY`。保持 `AGGREGATE_FUNCTIONS` frozenset 同步更新。
|
||||
|
||||
**KTD-7. Lookup/Rollup 作为独立 FieldType，不作为 formula 子类型。**
|
||||
新增 `FieldType.lookup`（已存在，扩展 config）和 `FieldType.rollup`（新增）。理由：1) 类型明确，UI 渲染清晰；2) 重算逻辑独立（lookup 不聚合，rollup 聚合）；3) 对齐飞书/Notion/Baserow 的字段类型划分。config 形状：`lookup: {relation_field_id, target_field_id}`；`rollup: {relation_field_id, target_field_id, aggregation: "sum"|"count"|"avg"|"min"|"max"|"unique"}`。
|
||||
|
||||
**KTD-8. AST 深度限制在 parser 层而非 engine 层。**
|
||||
在 `parse_formula()` 中递归计算 AST 深度，超过 15 抛 `FormulaDepthExceededError`。理由：1) 早失败，避免构建无效 DAG；2) validate-formula 端点直接返回错误；3) 不污染 engine 的拓扑排序逻辑。
|
||||
|
||||
**KTD-9. Webhook 入站用独立路由前缀 `/api/v1/bitable/webhooks/{token}`。**
|
||||
不用 JWT 认证，用 webhook-specific token（每规则唯一）。理由：1) 外部系统无法获取 JWT；2) token 可在 UI 重置；3) 与现有 `/api/v1/bitable` 内部端点隔离，安全边界清晰。token 存 automation config，恒定时间比较。
|
||||
|
||||
### Implementation Units
|
||||
|
||||
#### Phase 1: Storage & Model Foundation
|
||||
|
||||
### U1. Schema V3 Migration + Data Models Extension
|
||||
|
||||
**Goal:** 扩展 bitable 存储模型支持关联、自动化、跨表依赖追踪。新增 junction table、automation 表、依赖图表，扩展 Field/Record 模型。
|
||||
|
||||
**Requirements:** R5, R9, R16
|
||||
|
||||
**Dependencies:** 无（基础单元）
|
||||
|
||||
**Files:**
|
||||
- `src/agentkit/bitable/db.py` — bump `_SCHEMA_VERSION` 2→3，新增 ORM models（RelationLinkModel, AutomationModel, AutomationLogModel, CrossTableDepModel），新增 `_apply_v3_migration`
|
||||
- `src/agentkit/bitable/models.py` — 新增 `FieldType.rollup`、`AutomationTrigger`、`AutomationAction`、`AutomationRule`、`AutomationLog`、`RelationLink`、`CrossTableDep` Pydantic 模型；扩展 `Field.config` 文档说明 relation/rollup 配置形状
|
||||
- `src/agentkit/bitable/repository.py` — 新增 RelationLinks / Automations / AutomationLogs / CrossTableDeps CRUD 方法
|
||||
- `tests/unit/bitable/test_db.py` — V3 migration 测试
|
||||
- `tests/unit/bitable/test_models.py` — 新模型序列化测试
|
||||
|
||||
**Approach:**
|
||||
- `_apply_v3_migration(conn)` 创建 4 张新表（idempotent CREATE TABLE IF NOT EXISTS）：
|
||||
- `bitable_relation_links`：`(id, relation_field_id, source_record_id, target_record_id, created_at)` + 索引 `(relation_field_id, source_record_id)` 和 `(relation_field_id, target_record_id)` 用于双向查询
|
||||
- `bitable_automations`：`(id, table_id, name, trigger_config JSONB, action_config JSONB, enabled BOOL, created_at)` + 索引 `table_id`
|
||||
- `bitable_automation_logs`：`(id, automation_id, trigger_record_id, status, error_message, started_at, completed_at)` + 索引 `automation_id, started_at`
|
||||
- `bitable_cross_table_deps`：`(id, source_table_id, source_field_id, target_table_id, target_field_id, dep_type)` + 唯一约束 `(source_field_id, target_field_id)` + 索引 `target_field_id`（反向查询用）
|
||||
- `FieldType` 新增 `rollup = "rollup"`；`relation` 不新增（用 `lookup` field_type + `config.relation_field_id` 表示关联字段，对齐现有 lookup 设计但扩展 config）
|
||||
- `Field.config` 形状扩展：relation 字段 `{"relation_type": "one_to_one"|"one_to_many"|"many_to_many", "target_table_id": "...", "is_bidirectional": bool, "reverse_field_id": "..."}`；rollup 字段 `{"relation_field_id": "...", "target_field_id": "...", "aggregation": "sum"|"count"|"avg"|"min"|"max"|"unique"}`
|
||||
|
||||
**Patterns to follow:**
|
||||
- `_apply_v2_migration` 的 idempotent 模式（CREATE TABLE IF NOT EXISTS + ADD COLUMN IF NOT EXISTS）
|
||||
- 现有 ORM models 的 `__table_args__` schema 限定模式
|
||||
- Pydantic `model_config = ConfigDict(from_attributes=True)` 约定
|
||||
|
||||
**Test scenarios:**
|
||||
- **Happy path:** 全新 DB init → V3 schema 创建 4 张新表 + V2 表保留
|
||||
- **Migration:** V2 DB → init → V3 migration 应用，新表创建，现有数据不丢失
|
||||
- **Idempotent:** V3 DB → init → 不重复创建表
|
||||
- **Covers AE2 (junction table 创建部分).**
|
||||
- **Edge case:** meta 表 schema_version 正确更新为 3
|
||||
- **Integration:** repository 新增 CRUD 方法（create_relation_link, list_relation_links_by_source, etc.）读写正常
|
||||
|
||||
**Verification:** `pytest tests/unit/bitable/test_db.py tests/unit/bitable/test_models.py -x -q` 通过；V3 migration 在空 DB 和 V2 DB 上都幂等。
|
||||
|
||||
---
|
||||
|
||||
#### Phase 2: Formula Engine Expansion
|
||||
|
||||
### U2. Formula Function Expansion (10 → 40+)
|
||||
|
||||
**Goal:** 扩展公式函数从 10 个到 40+，覆盖日期、文本、逻辑、条件聚合、查找引用五大类别，保持 AST 白名单安全。
|
||||
|
||||
**Requirements:** R1, R3, R4
|
||||
|
||||
**Dependencies:** 无（与 U1 并行）
|
||||
|
||||
**Files:**
|
||||
- `src/agentkit/bitable/formula/functions/` — 新建包目录
|
||||
- `__init__.py` — 聚合所有子模块到 `FUNCTION_REGISTRY` 和 `AGGREGATE_FUNCTIONS`
|
||||
- `datetime.py` — DATE, DATEDIF, NETWORKDAYS, WEEKDAY, EOMONTH, TODAY, NOW, YEAR, MONTH, DAY, HOUR, MINUTE, SECOND, DATEADD, TIMESTAMP（15 个）
|
||||
- `text.py` — FIND, MID, REPLACE, SUBSTITUTE, UPPER, LOWER, TRIM, LEFT, RIGHT, REPT, TEXT, VALUE, STARTSWITH, ENDSWITH, SPLIT（15 个）
|
||||
- `logical.py` — IFS, SWITCH, IFERROR, ISBLANK, AND, OR, NOT, ISNUMBER, ISTEXT, ISDATE, COALESCE, NULL, TRUE, FALSE（14 个）
|
||||
- `aggregate.py` — SUMIF, COUNTIF, AVERAGEIF, SUMIFS, COUNTIFS（5 个条件聚合，迁移现有 SUM/AVG/COUNT/MIN/MAX）
|
||||
- `lookup.py` — LOOKUP, FILTER, MATCH, VLOOKUP, XLOOKUP（5 个，跨表引用支持在 U3 实现）
|
||||
- `src/agentkit/bitable/formula/functions.py` — 删除（迁移到包），保留兼容 re-export
|
||||
- `src/agentkit/bitable/formula/parser.py` — 新增 `FormulaDepthExceededError`，`parse_formula` 中计算 AST 深度，超 15 抛错；`_ALLOWED_NODES` 扩展支持 `ast.List`（用于 FILTER/MATCH 返回列表）和 `ast.Tuple`（用于多参数）
|
||||
- `src/agentkit/bitable/formula/engine.py` — `evaluate_all_for_record` 扩展支持列表类型返回值
|
||||
- `tests/unit/bitable/test_formula_parser.py` — 深度限制测试
|
||||
- `tests/unit/bitable/test_formula_engine.py` — 40+ 函数逐个测试
|
||||
- `tests/unit/bitable/test_functions_datetime.py` — 日期函数专项测试
|
||||
- `tests/unit/bitable/test_functions_text.py` — 文本函数专项测试
|
||||
- `tests/unit/bitable/test_functions_logical.py` — 逻辑函数专项测试
|
||||
- `tests/unit/bitable/test_functions_aggregate.py` — 条件聚合测试
|
||||
|
||||
**Approach:**
|
||||
- 日期函数处理 ISO 8601 字符串（现有 date 字段存储格式），NETWORKDAYS 用 Python `datetime.weekday()` 排除周末（不支持节假日表，留给 v3）
|
||||
- 文本函数对齐 Excel/Notion 命名（FIND/MID/REPLACE 与 Excel 一致）
|
||||
- 条件聚合 `SUMIF(range, criteria, [sum_range])` — range 是字段引用（列），criteria 是字符串/数字/表达式，sum_range 可选
|
||||
- LOOKUP/FILTER/MATCH 在 U3 实现跨表引用逻辑，本单元先实现单表内查找
|
||||
- AST 深度计算：递归 `_ast_depth(node)` 返回 `1 + max(child_depths)`，在 `parse_formula` 末尾检查
|
||||
- `FUNCTION_REGISTRY` 兼容：现有代码 `from agentkit.bitable.formula.functions import FUNCTION_REGISTRY` 通过 `__init__.py` re-export 保持不变
|
||||
|
||||
**Patterns to follow:**
|
||||
- 现有 `functions.py` 的 `FormulaResult: TypeAlias = str | int | float | None` 类型约定
|
||||
- 现有 `_sum`/`_avg` 的 None/空字符串过滤模式
|
||||
- 现有 `AGGREGATE_FUNCTIONS` frozenset 注册模式
|
||||
|
||||
**Test scenarios:**
|
||||
- **Happy path (per function):** 每个新函数至少 1 个 happy path 测试，验证返回值正确
|
||||
- **Covers AE3:** `=NETWORKDAYS({start_date}, {end_date})` 验证通过并返回工作日天数
|
||||
- **Edge cases:** 空输入（None/空字符串）处理；日期函数对非日期字符串报错；条件聚合无匹配返回 0
|
||||
- **Error path:** 未知函数仍抛 `UnknownFunctionError`；AST 深度超 15 抛 `FormulaDepthExceededError`
|
||||
- **Integration:** `parse_formula("=SUMIF({f1}, '>10')")` + `evaluate_ast` 全流程通过；validate-formula 端点对新函数返回 200
|
||||
- **Security:** 尝试 `=eval("code")` 仍被 `_SecurityVisitor` 拒绝；`=__import__('os')` 拒绝
|
||||
|
||||
**Verification:** `pytest tests/unit/bitable/test_functions_*.py tests/unit/bitable/test_formula_parser.py -x -q` 通过；40+ 函数全部在 `FUNCTION_REGISTRY` 注册。
|
||||
|
||||
---
|
||||
|
||||
### U3. Cross-Table Formula References
|
||||
|
||||
**Goal:** 支持跨表公式引用 `{relation_field}.{target_field}` 行级 + `[relation_field].[target_field]` 整列，依赖 U1 的 relation 基础设施。
|
||||
|
||||
**Requirements:** R2
|
||||
|
||||
**Dependencies:** U1, U2
|
||||
|
||||
**Files:**
|
||||
- `src/agentkit/bitable/formula/parser.py` — 扩展 `_FIELD_REF_RE` 支持 `.` 分隔的二级路径；`_substitute_field_refs` 返回 `field_mapping` 增加 `cross_table_refs: list[(safe_name, relation_field_id, target_field_id)]`
|
||||
- `src/agentkit/bitable/formula/engine.py` — `add_formula` 扩展 `_classify_refs` 识别跨表引用；`evaluate` 扩展 `relation_context` 参数；新增 `_resolve_cross_table_ref` 方法
|
||||
- `src/agentkit/bitable/formula/functions/lookup.py` — 实现 LOOKUP/FILTER/MATCH 的跨表逻辑（依赖 relation_context）
|
||||
- `src/agentkit/bitable/service.py` — `create_field` / `update_field` 中解析公式跨表引用，写入 `bitable_cross_table_deps`（KTD-3）
|
||||
- `tests/unit/bitable/test_formula_parser.py` — 跨表引用解析测试
|
||||
- `tests/unit/bitable/test_formula_engine.py` — 跨表公式求值测试
|
||||
|
||||
**Approach:**
|
||||
- parser 扩展正则：`_FIELD_REF_RE = re.compile(r"\{([a-zA-Z0-9_-]+)(?:\.([a-zA-Z0-9_-]+))?\}")`，第二组为可选 target_field
|
||||
- 整列引用 `[rel_field].[target_field]` 用单独正则 `_\[([a-zA-Z0-9_-]+)\]\.\[([a-zA-Z0-9_-]+)\]`，替换为 `__col_<rel>_<target>` Name 节点
|
||||
- engine `evaluate` 签名扩展：新增 `relation_context: dict[str, list[dict[str, object]]]` 参数，key 为 relation_field_id，value 为关联记录列表
|
||||
- service 层在创建/更新 formula 字段时，解析公式 AST 提取跨表引用，写入 `bitable_cross_table_deps`（dep_type: `formula`）
|
||||
- LOOKUP/FILTER/MATCH 函数接收 relation_context，通过 target_field_id 拉取值
|
||||
|
||||
**Patterns to follow:**
|
||||
- 现有 `_substitute_field_refs` 的 safe_name 映射模式
|
||||
- 现有 `_classify_refs` 的 aggregate vs row 分类模式
|
||||
|
||||
**Test scenarios:**
|
||||
- **Happy path:** `={rel_field}.{target_field}` 解析为跨表行级引用；`=[rel_field].[target_field]` 解析为整列引用
|
||||
- **Covers AE1 (跨表公式部分):** Orders.amount 修改 → Customers.total_amount（通过 rollup，U5 实现）跨表重算
|
||||
- **Edge case:** 自引用关联（rel_field 指向当前表）正常解析
|
||||
- **Error path:** relation_field 不存在 → 抛 `FormulaParseError`；target_field 不存在 → 抛 `FormulaParseError`
|
||||
- **Integration:** service.create_field 写入 cross_table_deps 记录；engine.evaluate 传入 relation_context 后正确求值
|
||||
|
||||
**Verification:** `pytest tests/unit/bitable/test_formula_parser.py tests/unit/bitable/test_formula_engine.py -x -q` 通过；跨表引用解析和求值正确。
|
||||
|
||||
---
|
||||
|
||||
#### Phase 3: Relations
|
||||
|
||||
### U4. Relation CRUD + Junction Table Management
|
||||
|
||||
**Goal:** 实现单向/双向/多对多/自引用关联字段的创建、更新、删除，管理 junction table 生命周期。
|
||||
|
||||
**Requirements:** R5, R7, R8
|
||||
|
||||
**Dependencies:** U1
|
||||
|
||||
**Files:**
|
||||
- `src/agentkit/bitable/service.py` — 新增 `create_relation_field`、`update_relation_link`（添加/删除关联记录）、`delete_relation_field`（级联处理）方法
|
||||
- `src/agentkit/bitable/repository.py` — 新增 `add_relation_link`、`remove_relation_link`、`list_relation_links`、`list_reverse_relation_links` 方法
|
||||
- `src/agentkit/server/routes/bitable.py` — 新增端点：`POST /tables/{id}/fields`（扩展支持 relation field_type）、`POST /fields/{id}/links`（添加关联）、`DELETE /fields/{id}/links/{record_id}`（删除关联）
|
||||
- `tests/unit/bitable/test_service.py` — relation CRUD 测试
|
||||
- `tests/unit/bitable/test_routes.py` — relation 端点测试
|
||||
|
||||
**Approach:**
|
||||
- `create_relation_field(table_id, name, target_table_id, relation_type, is_bidirectional)`：
|
||||
- one_to_one / one_to_many：关联值存记录 JSONB `values[relation_field_id]` = `["rec_id"]` 或 `["rec_id1", "rec_id2"]`
|
||||
- many_to_many：创建独立 junction table 引用（KTD-1），字段 config 存 `relation_type` + `target_table_id`
|
||||
- is_bidirectional=True：在 target_table 自动创建反向字段，config 互相引用 `reverse_field_id`
|
||||
- 自引用：target_table_id = 当前 table_id，正常流程
|
||||
- `update_relation_link(field_id, source_record_id, target_record_ids)`：many_to_many 操作 `bitable_relation_links`；其他类型更新记录 JSONB
|
||||
- `delete_relation_field(field_id, force)`：扩展 `FieldDependencyError` 检查——查找依赖该 relation 的 lookup/rollup/跨表公式字段；force=True 时级联删除依赖字段的 config
|
||||
|
||||
**Patterns to follow:**
|
||||
- 现有 `delete_field` 的 `FieldDependencyError` + force 级联模式（service.py:234-282）
|
||||
- 现有 `_check_table_ownership` IDOR 模式
|
||||
- 现有 `create_field` 的 config 验证模式
|
||||
|
||||
**Test scenarios:**
|
||||
- **Happy path:** 创建单向关联（A→B）；创建双向关联（A↔B，自动建反向字段）；创建多对多关联（junction table 生成）
|
||||
- **Covers AE2 (关联创建部分):** Students↔Courses 多对多创建，双向字段生成
|
||||
- **Covers F2:** 多对多关联创建流程完整验证
|
||||
- **Edge case:** 自引用关联（table A 关联自身）正常工作
|
||||
- **Error path:** target_table_id 不存在 → 400；删除有依赖的 relation 字段未 force → `FieldDependencyError`
|
||||
- **Integration:** 创建关联字段 → 添加关联记录 → 查询关联记录列表 → 删除关联
|
||||
|
||||
**Verification:** `pytest tests/unit/bitable/test_service.py tests/unit/bitable/test_routes.py -x -q` 通过；三种关联类型 CRUD 完整。
|
||||
|
||||
---
|
||||
|
||||
### U5. Lookup & Rollup Fields
|
||||
|
||||
**Goal:** 实现 Lookup 字段拉取关联记录字段值，Rollup 字段对关联记录聚合（sum/count/avg/min/max/unique）。
|
||||
|
||||
**Requirements:** R6, R8
|
||||
|
||||
**Dependencies:** U1, U4
|
||||
|
||||
**Files:**
|
||||
- `src/agentkit/bitable/service.py` — 新增 `create_lookup_field`、`create_rollup_field` 方法；扩展 `_trigger_recalc_for_affected_fields` 识别 lookup/rollup 字段
|
||||
- `src/agentkit/bitable/recalc_worker.py` — 扩展 `process_task` 处理 lookup/rollup 字段类型；新增 `_resolve_lookup`、`_resolve_rollup` 方法
|
||||
- `src/agentkit/bitable/formula/engine.py` — 扩展支持 lookup/rollup 字段作为"虚拟公式"注册到 DAG
|
||||
- `tests/unit/bitable/test_service.py` — lookup/rollup 创建测试
|
||||
- `tests/unit/bitable/test_recalc.py` — lookup/rollup 重算测试
|
||||
|
||||
**Approach:**
|
||||
- lookup 字段 config: `{"relation_field_id": "...", "target_field_id": "..."}`，重算时通过 relation 找到关联记录，拉取 target_field 值
|
||||
- rollup 字段 config: `{"relation_field_id": "...", "target_field_id": "...", "aggregation": "sum"}`，重算时拉取关联记录的 target_field 列表，应用聚合函数
|
||||
- recalc_worker 扩展：`process_task` 检查 field.field_type，formula → 现有逻辑；lookup → `_resolve_lookup`；rollup → `_resolve_rollup`
|
||||
- `_resolve_lookup(field, record)`：查 `bitable_relation_links`（多对多）或记录 JSONB（一对多）获取关联记录 ID 列表 → `get_records_batch` → 提取 target_field 值 → 返回（单值或列表）
|
||||
- `_resolve_rollup(field, record)`：同 lookup 拉取值列表 → 应用 aggregation 函数 → 返回标量
|
||||
- 跨表依赖：lookup/rollup 字段创建时写入 `bitable_cross_table_deps`（dep_type: `lookup` / `rollup`）
|
||||
|
||||
**Patterns to follow:**
|
||||
- 现有 `process_task` 的 formula 处理流程（recalc_worker.py:177-235）
|
||||
- 现有 `get_column_values` 批量拉取模式
|
||||
- 现有 `FieldDependencyError` 级联模式（R8）
|
||||
|
||||
**Test scenarios:**
|
||||
- **Happy path:** 创建 lookup 字段拉取关联记录的 text 字段；创建 rollup 字段对关联记录 number 字段求和
|
||||
- **Covers AE2 (lookup/rollup 部分):** Students 创建 lookup 拉取 Course.credit；创建 rollup 统计总学分
|
||||
- **Covers AE1:** Orders.amount 修改 → Customers.total_amount（rollup SUM）跨表重算
|
||||
- **Edge case:** 关联记录为空时 lookup 返回 None，rollup 返回 0；unique 聚合去重
|
||||
- **Error path:** relation_field 已删除 → 字段 config 标记 error；target_field 已删除 → 同上
|
||||
- **Integration:** 创建关联 → 添加关联记录 → 创建 rollup → 验证重算结果正确
|
||||
|
||||
**Verification:** `pytest tests/unit/bitable/test_service.py tests/unit/bitable/test_recalc.py -x -q` 通过；lookup/rollup 重算结果正确。
|
||||
|
||||
---
|
||||
|
||||
#### Phase 4: Cross-Table Recalc
|
||||
|
||||
### U6. Cross-Table Dependency Graph + Recalc Worker Extension
|
||||
|
||||
**Goal:** 构建跨表字段级依赖图，扩展 recalc worker 支持跨表重算，异步不阻塞原始写入。
|
||||
|
||||
**Requirements:** R10, R11, R12
|
||||
|
||||
**Dependencies:** U1, U3, U5
|
||||
|
||||
**Files:**
|
||||
- `src/agentkit/bitable/repository.py` — 新增 `find_cross_table_dependents(target_table_id, target_field_id)` 方法
|
||||
- `src/agentkit/bitable/recalc_worker.py` — 扩展 `_run` 循环处理跨表任务；新增 `_enqueue_cross_table_recalc` 方法；`process_task` 扩展跨表上下文加载
|
||||
- `src/agentkit/bitable/service.py` — `_trigger_recalc_for_affected_fields` 扩展：查跨表依赖图，enqueue 跨表任务
|
||||
- `src/agentkit/bitable/models.py` — `RecalcTask` 新增 `is_cross_table: bool` 字段
|
||||
- `src/agentkit/bitable/db.py` — `RecalcQueueModel` 新增 `is_cross_table` 列（V3 migration 一部分）
|
||||
- `tests/unit/bitable/test_recalc.py` — 跨表重算测试
|
||||
|
||||
**Approach:**
|
||||
- `RecalcTask.is_cross_table` 标记跨表任务（默认 False）
|
||||
- service `_trigger_recalc_for_affected_fields(table_id, record_id, changed_field_ids)`：
|
||||
1. 查本表公式字段依赖 → enqueue 本表 recalc（现有逻辑）
|
||||
2. 查 `bitable_cross_table_deps` where `target_table_id = table_id AND target_field_id IN changed_field_ids` → 对每个依赖字段，找到依赖它的记录（通过 relation_links）→ enqueue 跨表 recalc 任务
|
||||
- worker `process_task` 扩展：if `task.is_cross_table` → 加载 relation_context（关联记录列表）→ 传给 engine.evaluate
|
||||
- 跨表依赖图构建在字段创建/更新时同步（U3 service hook + U5 lookup/rollup 创建 hook）
|
||||
- 拓扑排序扩展：跨表依赖也加入 DAG，循环检测覆盖跨表
|
||||
|
||||
**Patterns to follow:**
|
||||
- 现有 `claim_recalc_tasks` 的 `FOR UPDATE SKIP LOCKED` 模式
|
||||
- 现有 `_sort_by_topological_order` 的 Kahn 算法
|
||||
- 现有 `reset_stale_recalc_tasks` 的 crash recovery 模式
|
||||
|
||||
**Test scenarios:**
|
||||
- **Happy path:** 修改表 A 记录 → 表 B 依赖该记录的 rollup 字段异步重算并更新
|
||||
- **Covers AE1 完整:** Orders.amount 修改 → Customers.total_amount 跨表重算
|
||||
- **Covers F1:** 跨表公式重算流程完整验证
|
||||
- **Edge case:** 无跨表依赖时不 enqueue 跨表任务；循环依赖检测（A 依赖 B，B 依赖 A）抛 `CircularReferenceError`
|
||||
- **Error path:** 跨表任务失败 → 状态 error + error_message；stale 任务被 reaper 重置
|
||||
- **Integration:** 创建关联 → 创建 rollup → 修改源记录 → 验证目标记录异步更新（轮询 recalc 完成状态）
|
||||
|
||||
**Verification:** `pytest tests/unit/bitable/test_recalc.py -x -q` 通过；跨表重算异步且正确。
|
||||
|
||||
---
|
||||
|
||||
#### Phase 5: Automation
|
||||
|
||||
### U7. Automation Engine (Triggers + Actions + Webhook + Retry + Logging)
|
||||
|
||||
**Goal:** 实现 3 种触发器、5 种动作、Webhook 入站、失败重试 3 次、执行日志可查。
|
||||
|
||||
**Requirements:** R13, R14, R15, R16, R17
|
||||
|
||||
**Dependencies:** U1
|
||||
|
||||
**Files:**
|
||||
- `src/agentkit/bitable/automation/` — 新建包
|
||||
- `__init__.py`
|
||||
- `engine.py` — `AutomationEngine` 类：`match_triggers(table_id, record_id, change_type, changed_fields)`、`execute_action(rule, trigger_record)`
|
||||
- `triggers.py` — 触发器匹配逻辑（record_created / record_updated / record_deleted）
|
||||
- `actions.py` — 5 种动作执行器（create_record / update_record / send_webhook / send_email / send_message）
|
||||
- `webhook.py` — Webhook 入站端点处理 + 出站 HTTP 调用（含 SSRF guard，复用 `ingestion/excel.py` 的 SSRF 防护）
|
||||
- `src/agentkit/bitable/service.py` — record mutation hook 调用 `AutomationEngine.match_triggers`；`create_automation_rule`、`list_automation_rules`、`update_automation_rule`、`delete_automation_rule`、`list_automation_logs` 方法
|
||||
- `src/agentkit/server/routes/bitable.py` — 新增端点：`POST /tables/{id}/automations`、`GET /tables/{id}/automations`、`PATCH /automations/{id}`、`DELETE /automations/{id}`、`GET /automations/{id}/logs`、`POST /webhooks/{token}`（入站 webhook，无 JWT 认证）
|
||||
- `tests/unit/bitable/test_automation.py` — 自动化引擎测试
|
||||
- `tests/unit/bitable/test_routes.py` — 自动化端点测试
|
||||
|
||||
**Approach:**
|
||||
- `AutomationEngine` 在 service record mutation hook 后同步调用 `match_triggers`，匹配到的规则创建独立 asyncio task（KTD-4）
|
||||
- 触发器 config 形状：`{"type": "record_updated", "field_ids": ["f1"]}`（field_ids 可选，不指定则任何字段变更触发）
|
||||
- 动作 config 形状：`{"type": "send_webhook", "url": "https://...", "method": "POST", "headers": {...}}`；`{"type": "create_record", "target_table_id": "...", "values": {...}}`；`{"type": "send_email", "to": "...", "subject": "...", "body": "..."}`；`{"type": "send_message", "channel": "...", "message": "..."}`
|
||||
- Webhook 出站复用 `ingestion/excel.py` 的 `_is_private_ip` / `_validate_url` SSRF 防护
|
||||
- Webhook 入站：`POST /api/v1/bitable/webhooks/{token}` → 查 automation by token（恒定时间比较）→ 匹配 trigger → 执行 action
|
||||
- 重试：exponential backoff（1s/2s/4s），3 次失败后 `AutomationLog.status = "error"`
|
||||
- `send_message` 集成现有 `bus/` MemoryBus 或 RedisBus（视配置）
|
||||
- `send_email` 用 `aiosmtplib`（如已安装）或 stdlib `smtplib` via `asyncio.to_thread`
|
||||
|
||||
**Patterns to follow:**
|
||||
- 现有 `RecalcWorker` 的 asyncio task 生命周期管理（start/stop/cancel）
|
||||
- 现有 `ingestion/excel.py` 的 SSRF 防护（`_is_private_ip`, URL scheme 验证）
|
||||
- 现有 `hmac.compare_digest` 恒定时间比较模式
|
||||
- 现有 route 的 IDOR ownership 检查模式
|
||||
|
||||
**Test scenarios:**
|
||||
- **Happy path:** 创建自动化规则（record_updated + send_webhook）→ 修改记录 → webhook 被调用
|
||||
- **Covers AE4 完整:** Tasks.status 改为"已完成" → webhook 触发 → 失败重试 3 次 → 标记 error
|
||||
- **Covers F4:** 自动化规则执行流程完整验证
|
||||
- **Edge case:** trigger 指定 field_ids 时，非指定字段变更不触发；webhook 出站 URL 为私网 IP → 拒绝（SSRF guard）
|
||||
- **Error path:** 动作执行失败 → 重试 3 次 → AutomationLog 记录 error + error_message；webhook token 无效 → 404
|
||||
- **Integration:** 创建规则 → 触发 → 查询日志 → 日志显示成功/失败状态
|
||||
|
||||
**Verification:** `pytest tests/unit/bitable/test_automation.py tests/unit/bitable/test_routes.py -x -q` 通过；5 种动作 + 重试 + 日志完整。
|
||||
|
||||
---
|
||||
|
||||
#### Phase 6: Agent Integration
|
||||
|
||||
### U8. BitableTool Agent Actions + REST API + Frontend
|
||||
|
||||
**Goal:** BitableTool 扩展 4 个新 action，实现 agent 生成公式/自动化 + 创建关联/Rollup 的差异化能力，配套前端组件。
|
||||
|
||||
**Requirements:** R18, R19, R20
|
||||
|
||||
**Dependencies:** U1, U2, U3, U4, U5, U6, U7
|
||||
|
||||
**Files:**
|
||||
- `src/agentkit/tools/bitable_tool.py` — 新增 4 个 action：`generate_formula`、`generate_automation`、`create_relation`、`create_rollup`；新增 `undo` action（撤销最近创建）
|
||||
- `src/agentkit/server/routes/bitable.py` — 新增端点：`POST /tables/{id}/generate-formula`（LLM 生成 + validate + 创建）、`POST /tables/{id}/generate-automation`、`POST /undo/{undo_token}`
|
||||
- `src/agentkit/server/frontend/src/api/bitable.ts` — 新增 API 方法：`generateFormula`、`generateAutomation`、`createRelation`、`createRollup`、`undoAction`
|
||||
- `src/agentkit/server/frontend/src/components/bitable/RelationFieldEditor.vue` — 新建：关联字段配置组件（选择目标表、关联类型、双向开关）
|
||||
- `src/agentkit/server/frontend/src/components/bitable/RollupFieldEditor.vue` — 新建：Rollup 字段配置组件（选择关联字段、目标字段、聚合函数）
|
||||
- `src/agentkit/server/frontend/src/components/bitable/AutomationRuleList.vue` — 新建：自动化规则列表 + 表单式编辑器
|
||||
- `src/agentkit/server/frontend/src/components/bitable/FormulaGeneratorModal.vue` — 新建：自然语言生成公式 modal
|
||||
- `src/agentkit/server/frontend/src/stores/bitable.ts` — 扩展 state：`automations`、`automationLogs`；actions：`generateFormula`、`createRelation` 等
|
||||
- `tests/unit/bitable/test_bitable_tool.py` — 新 action 测试
|
||||
|
||||
**Approach:**
|
||||
- `generate_formula(table_id, description, field_name)`：
|
||||
1. 调用 LLM（通过现有 `llm/` 网关）将 description + 表 schema 转为公式表达式
|
||||
2. 调用 `validate-formula` 端点验证语法（R20 复用）
|
||||
3. 验证通过 → `create_field(field_type="formula", config={"formula_expr": ...})`
|
||||
4. 返回 `{success, field_id, undo_token}`
|
||||
5. undo_token 存 Redis（5 分钟 TTL），undo 时删除字段
|
||||
- `generate_automation(table_id, description)`：类似流程，LLM 生成 trigger_config + action_config
|
||||
- `create_relation(table_id, name, target_table_id, relation_type, is_bidirectional)`：调用 U4 的 service 方法
|
||||
- `create_rollup(table_id, name, relation_field_id, target_field_id, aggregation)`：调用 U5 的 service 方法
|
||||
- LLM prompt 工程：传入表字段 schema + 用户描述，要求 LLM 返回严格 JSON `{formula_expr: "..."}` 或 `{trigger: {...}, action: {...}}`
|
||||
- 前端组件遵循现有 Ant Design Vue 4 风格，`FieldConfigForm.vue` 扩展支持 relation/rollup 类型配置
|
||||
|
||||
**Patterns to follow:**
|
||||
- 现有 BitableTool 的 handler dispatch + httpx.AsyncClient 模式（bitable_tool.py:175-186）
|
||||
- 现有 `_create_view` / `_update_field` 的 REST 调用模式
|
||||
- 现有 `FieldConfigForm.vue` 的表单布局
|
||||
- 现有 Pinia store 的 actions 模式
|
||||
|
||||
**Test scenarios:**
|
||||
- **Happy path:** `generate_formula(description="计算总价 = 单价 × 数量")` → LLM 生成 `={price} * {quantity}` → validate 通过 → 创建字段
|
||||
- **Covers AE5 完整:** Agent 生成公式流程端到端验证
|
||||
- **Covers F3:** Agent 生成公式流程完整验证
|
||||
- **Edge case:** LLM 生成无效公式 → validate 失败 → 返回错误不创建字段；undo 在 5 分钟内有效，超时失效
|
||||
- **Error path:** LLM 调用失败 → 返回错误；表不存在 → 404
|
||||
- **Integration:** agent 调用 → 生成 → 创建字段 → recalc 触发 → 撤销 → 字段删除
|
||||
- **Frontend smoke:** RelationFieldEditor 选择目标表 + 关联类型 → 保存 → 字段创建；FormulaGeneratorModal 输入描述 → 生成 → 字段创建
|
||||
|
||||
**Verification:** `pytest tests/unit/bitable/test_bitable_tool.py -x -q` 通过；`cd src/agentkit/server/frontend && npm run typecheck` 通过；4 个新 action 端到端可用。
|
||||
|
||||
---
|
||||
|
||||
### Verification Contract
|
||||
|
||||
**Per-unit verification (each U-ID):**
|
||||
- 每个单元的 `pytest tests/unit/bitable/test_*.py -x -q` 必须通过
|
||||
- 涉及前端改动的单元（U8）需 `npm run typecheck` 通过
|
||||
- 涉及 DB migration 的单元（U1）需在空 DB 和 V2 DB 上都幂等
|
||||
|
||||
**Integration verification (after all units):**
|
||||
- `pytest tests/unit/bitable/ -x -q` 全部通过
|
||||
- `ruff check src/agentkit/bitable/ src/agentkit/tools/bitable_tool.py && ruff format src/agentkit/bitable/` 无错误
|
||||
- 手动 smoke test（可选）：启动服务 → 创建两表 → 建立多对多关联 → 创建 rollup → 修改源记录 → 验证跨表重算
|
||||
|
||||
**Acceptance Examples enforcement:**
|
||||
- AE1 (跨表重算): U5 + U6 实现并测试
|
||||
- AE2 (多对多): U1 + U4 + U5 实现并测试
|
||||
- AE3 (公式扩展): U2 实现并测试
|
||||
- AE4 (自动化): U7 实现并测试
|
||||
- AE5 (Agent 生成): U8 实现并测试
|
||||
|
||||
**Behavior change verification:**
|
||||
- 公式引擎扩展是增量，现有 10 个函数行为不变（向后兼容）
|
||||
- 现有 lookup 字段类型扩展 config，旧 lookup 字段（无 relation_field_id）仍可读
|
||||
- recalc worker 扩展，现有单表 recalc 流程不变
|
||||
|
||||
### Definition of Done
|
||||
|
||||
- [ ] U1-U8 全部实现，单元测试通过
|
||||
- [ ] `pytest tests/unit/bitable/ -x -q` 全绿
|
||||
- [ ] `ruff check src/agentkit/bitable/ && ruff format src/agentkit/bitable/` 无错误
|
||||
- [ ] 前端 `npm run typecheck` 通过（U8 部分）
|
||||
- [ ] Schema V3 migration 在空 DB 和 V2 DB 上幂等
|
||||
- [ ] 现有 bitable 功能无回归（现有测试全绿）
|
||||
- [ ] AE1-AE5 验收示例对应的测试场景全部通过
|
||||
- [ ] BitableTool 新 action 通过 REST API 可用
|
||||
- [ ] 公式引擎 40+ 函数在 `FUNCTION_REGISTRY` 注册
|
||||
- [ ] 跨表依赖图正确构建和更新
|
||||
- [ ] 自动化引擎 3 触发器 + 5 动作 + 重试 + 日志完整
|
||||
|
||||
### Open Questions
|
||||
|
||||
- **email 发送实现**：`aiosmtplib` 是否已在依赖中？如无，用 stdlib `smtplib` via `asyncio.to_thread`（ponytail: 不引入新依赖）。实现时检查 `pyproject.toml`。
|
||||
- **LLM 网关调用细节**：`generate_formula` 的 LLM 调用走现有 `llm/` 网关的哪个接口？实现时参考 `chat/handler.py` 的 LLM 调用模式。
|
||||
- **前端组件 SSR**：Tauri 桌面端是否需要同步更新？v2 聚焦 Web，Tauri 兼容性留给实现时验证。
|
||||
|
||||
### Risks & Dependencies
|
||||
|
||||
**Risks:**
|
||||
- **跨表重算性能**：跨表依赖图可能在大数据量下产生大量 recalc 任务。缓解：`is_cross_table` 标记 + 批量 enqueue + worker 限流（现有 `claim_recalc_tasks(limit=10)`）。
|
||||
- **LLM 生成公式质量**：LLM 可能生成语法正确但语义错误的公式。缓解：validate-formula 仅验证语法；agent 返回结果包含 `formula_expr` 供用户审查；undo 机制。
|
||||
- **多对多 junction table 膨胀**：高频关联操作可能产生大量 junction 记录。缓解：定期清理孤立链接（v3 优化）；索引覆盖查询路径。
|
||||
- **Schema migration 风险**：V3 migration 在生产数据上失败。缓解：idempotent CREATE IF NOT EXISTS；现有 V2 migration 模式已验证；无生产数据（grounding 确认）。
|
||||
|
||||
**Dependencies:**
|
||||
- 现有 `llm/` 网关（U8 LLM 调用）
|
||||
- 现有 `bus/` 消息总线（U7 send_message 动作）
|
||||
- 现有 `ingestion/excel.py` SSRF 防护（U7 webhook 出站）
|
||||
- 现有 `hmac.compare_digest`（U7 webhook token 比较）
|
||||
|
||||
### System-Wide Impact
|
||||
|
||||
- **后端**：bitable 子系统独立 schema 扩展，不影响其他模块；新增 `automation/` 包；`tools/bitable_tool.py` 扩展
|
||||
- **前端**：新增 4 个组件，扩展 store 和 API client；现有 `FieldConfigForm.vue` 扩展（向后兼容）
|
||||
- **数据库**：bitable schema V3，4 张新表；不影响其他 schema
|
||||
- **部署**：无新依赖（ponytail: 复用现有）；无新环境变量（webhook token 存 DB）
|
||||
- **文档**：无需更新 AGENTS.md（bitable 模块自包含）
|
||||
|
||||
### Phased Delivery
|
||||
|
||||
| 阶段 | 单元 | 依赖 | 可交付价值 |
|
||||
|---|---|---|---|
|
||||
| 1 | U1 | 无 | 存储基础 + 数据模型 |
|
||||
| 2 | U2, U4 | U1 | 公式 40+ 函数 + 关联 CRUD |
|
||||
| 3 | U3, U5 | U1, U2, U4 | 跨表公式 + Lookup/Rollup |
|
||||
| 4 | U6 | U1, U3, U5 | 跨表重算 |
|
||||
| 5 | U7 | U1 | 自动化引擎 |
|
||||
| 6 | U8 | 全部 | Agent 集成 + 前端 |
|
||||
|
||||
**执行方向提示：** 每个单元实现后立即运行对应测试；U1 migration 优先验证幂等性；U2 函数扩展按类别分模块并行实现；U6 跨表重算需端到端集成测试。
|
||||
|
|
@ -0,0 +1,349 @@
|
|||
"""Automation engine — trigger evaluation, action execution, retry, logging (U7, R13-R17).
|
||||
|
||||
Lifecycle:
|
||||
engine = AutomationEngine(service)
|
||||
await engine.start() # starts background task
|
||||
...
|
||||
await engine.stop()
|
||||
|
||||
Triggers (R13):
|
||||
- record_created: fires when a record is created in the watched table
|
||||
- record_updated: fires when a record is updated (optionally filtered by field_ids)
|
||||
- record_deleted: fires when a record is deleted
|
||||
|
||||
Actions (R14):
|
||||
- create_record: insert a new record in a target table
|
||||
- update_record: update fields on the triggering record (or a related record)
|
||||
- send_webhook: POST to a URL with the trigger payload
|
||||
- send_email: placeholder — logs the email content (SMTP integration is app-level)
|
||||
- send_message: placeholder — logs the message (channel integration is app-level)
|
||||
|
||||
Retry: each action is retried up to 3 times with exponential backoff (1s, 2s, 4s).
|
||||
All executions are logged to ``automation_logs`` (R17).
|
||||
|
||||
Webhook inbound (R16): ``handle_webhook(token, payload)`` matches a rule by its
|
||||
``webhook_token`` and fires its action with the payload as trigger context.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import secrets
|
||||
from datetime import datetime, timezone
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import httpx
|
||||
|
||||
from agentkit.bitable.ingestion.excel import _assert_safe_host
|
||||
from agentkit.bitable.models import (
|
||||
AutomationActionType,
|
||||
AutomationStatus,
|
||||
AutomationTriggerType,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from agentkit.bitable.service import BitableService
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_MAX_RETRIES = 3
|
||||
_RETRY_DELAYS = [1.0, 2.0, 4.0] # seconds, exponential backoff
|
||||
_WEBHOOK_TIMEOUT = 10.0 # seconds
|
||||
|
||||
|
||||
class AutomationEngine:
|
||||
"""Background engine that evaluates triggers and executes actions.
|
||||
|
||||
The engine itself is triggerless — it exposes ``fire_trigger`` which
|
||||
the service layer calls after record writes. Actions execute in an
|
||||
isolated asyncio task (KTD-4) so failures don't block the caller.
|
||||
"""
|
||||
|
||||
def __init__(self, service: BitableService) -> None:
|
||||
self._service = service
|
||||
self._repo = service.repo
|
||||
self._task: asyncio.Task[None] | None = None
|
||||
self._queue: asyncio.Queue[dict[str, Any]] = asyncio.Queue()
|
||||
self._stop_event = asyncio.Event()
|
||||
|
||||
async def start(self) -> None:
|
||||
"""Start the background action executor."""
|
||||
self._stop_event.clear()
|
||||
self._task = asyncio.create_task(self._run(), name="automation-engine")
|
||||
logger.info("AutomationEngine started")
|
||||
|
||||
async def stop(self) -> None:
|
||||
"""Gracefully stop the engine."""
|
||||
self._stop_event.set()
|
||||
if self._task is not None:
|
||||
self._task.cancel()
|
||||
try:
|
||||
await self._task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self._task = None
|
||||
logger.info("AutomationEngine stopped")
|
||||
|
||||
async def _run(self) -> None:
|
||||
"""Main loop — process action executions from the queue."""
|
||||
while not self._stop_event.is_set():
|
||||
try:
|
||||
job = await asyncio.wait_for(self._queue.get(), timeout=1.0)
|
||||
await self._execute_with_retry(job)
|
||||
except asyncio.TimeoutError:
|
||||
continue
|
||||
except asyncio.CancelledError:
|
||||
break
|
||||
except Exception:
|
||||
logger.exception("AutomationEngine error in main loop")
|
||||
|
||||
async def fire_trigger(
|
||||
self,
|
||||
table_id: str,
|
||||
trigger_type: AutomationTriggerType,
|
||||
record_id: str | None = None,
|
||||
changed_field_ids: list[str] | None = None,
|
||||
record_values: dict[str, object] | None = None,
|
||||
) -> int:
|
||||
"""Evaluate trigger rules for a table event and enqueue matching actions.
|
||||
|
||||
Returns the number of rules matched (not necessarily executed yet —
|
||||
actions run asynchronously).
|
||||
|
||||
Args:
|
||||
table_id: The table where the event occurred.
|
||||
trigger_type: create / update / delete.
|
||||
record_id: The affected record (None for bulk operations).
|
||||
changed_field_ids: For record_updated — only fire if the rule
|
||||
watches these fields (empty/None means any field).
|
||||
record_values: Snapshot of the record's values (for action context).
|
||||
"""
|
||||
rules = await self._repo.list_automations_for_trigger(
|
||||
table_id=table_id, trigger_type=trigger_type.value
|
||||
)
|
||||
matched = 0
|
||||
for rule in rules:
|
||||
if not rule.enabled:
|
||||
continue
|
||||
# For record_updated with field_ids filter, check overlap
|
||||
if trigger_type == AutomationTriggerType.record_updated:
|
||||
watched = rule.trigger_config.get("field_ids", [])
|
||||
if watched and changed_field_ids:
|
||||
if not set(watched) & set(changed_field_ids):
|
||||
continue # No overlap — skip
|
||||
|
||||
matched += 1
|
||||
await self._queue.put(
|
||||
{
|
||||
"automation_id": rule.id,
|
||||
"table_id": table_id,
|
||||
"trigger_type": trigger_type.value,
|
||||
"record_id": record_id,
|
||||
"record_values": record_values or {},
|
||||
"action_config": rule.action_config,
|
||||
}
|
||||
)
|
||||
return matched
|
||||
|
||||
async def handle_webhook(self, token: str, payload: dict[str, Any]) -> bool:
|
||||
"""Handle an inbound webhook (R16).
|
||||
|
||||
Returns True if a rule matched the token and was executed.
|
||||
"""
|
||||
rule = await self._repo.get_automation_by_webhook_token(token)
|
||||
if rule is None or not rule.enabled:
|
||||
return False
|
||||
await self._queue.put(
|
||||
{
|
||||
"automation_id": rule.id,
|
||||
"table_id": rule.table_id,
|
||||
"trigger_type": "webhook",
|
||||
"record_id": None,
|
||||
"record_values": payload,
|
||||
"action_config": rule.action_config,
|
||||
}
|
||||
)
|
||||
return True
|
||||
|
||||
async def _execute_with_retry(self, job: dict[str, Any]) -> None:
|
||||
"""Execute an action with up to 3 retries (R15)."""
|
||||
automation_id = job["automation_id"]
|
||||
record_id = job.get("record_id")
|
||||
last_error: str | None = None
|
||||
|
||||
for attempt in range(1, _MAX_RETRIES + 1):
|
||||
log_id = await self._repo.create_automation_log(
|
||||
automation_id=automation_id,
|
||||
trigger_record_id=record_id,
|
||||
status=AutomationStatus.retrying,
|
||||
attempt=attempt,
|
||||
)
|
||||
try:
|
||||
await self._execute_action(job)
|
||||
await self._repo.update_automation_log(
|
||||
log_id=log_id,
|
||||
status=AutomationStatus.success,
|
||||
completed_at=datetime.now(timezone.utc),
|
||||
)
|
||||
return
|
||||
except Exception as e:
|
||||
last_error = str(e)[:500]
|
||||
logger.warning(
|
||||
"AutomationEngine: attempt %d failed for rule %s: %s",
|
||||
attempt,
|
||||
automation_id,
|
||||
last_error,
|
||||
)
|
||||
await self._repo.update_automation_log(
|
||||
log_id=log_id,
|
||||
status=AutomationStatus.error,
|
||||
error_message=last_error,
|
||||
completed_at=datetime.now(timezone.utc),
|
||||
)
|
||||
if attempt < _MAX_RETRIES:
|
||||
await asyncio.sleep(_RETRY_DELAYS[attempt - 1])
|
||||
|
||||
logger.error(
|
||||
"AutomationEngine: rule %s failed after %d attempts: %s",
|
||||
automation_id,
|
||||
_MAX_RETRIES,
|
||||
last_error,
|
||||
)
|
||||
|
||||
async def _execute_action(self, job: dict[str, Any]) -> None:
|
||||
"""Dispatch to the correct action handler based on action_config type."""
|
||||
action_config = job["action_config"]
|
||||
action_type = str(action_config.get("type", ""))
|
||||
record_values = job.get("record_values", {})
|
||||
|
||||
if action_type == AutomationActionType.create_record.value:
|
||||
await self._action_create_record(action_config, record_values)
|
||||
elif action_type == AutomationActionType.update_record.value:
|
||||
await self._action_update_record(action_config, job.get("record_id"), record_values)
|
||||
elif action_type == AutomationActionType.send_webhook.value:
|
||||
await self._action_send_webhook(action_config, job)
|
||||
elif action_type == AutomationActionType.send_email.value:
|
||||
self._action_send_email(action_config, job)
|
||||
elif action_type == AutomationActionType.send_message.value:
|
||||
self._action_send_message(action_config, job)
|
||||
else:
|
||||
raise ValueError(f"Unknown action type: {action_type}")
|
||||
|
||||
async def _action_create_record(
|
||||
self, config: dict[str, Any], trigger_values: dict[str, object]
|
||||
) -> None:
|
||||
"""Create a record in a target table (R14 create_record).
|
||||
|
||||
Config: ``{target_table_id, values: {field_id: value | "@trigger.field_id"}}``
|
||||
``@trigger.X`` placeholders are resolved from the triggering record's values.
|
||||
"""
|
||||
target_table_id = str(config.get("target_table_id", ""))
|
||||
if not target_table_id:
|
||||
raise ValueError("create_record action missing target_table_id")
|
||||
raw_values: dict[str, object] = config.get("values", {}) # type: ignore[assignment]
|
||||
resolved: dict[str, str | int | float | None] = {}
|
||||
for k, v in raw_values.items():
|
||||
if isinstance(v, str) and v.startswith("@trigger."):
|
||||
src_field = v[len("@trigger.") :]
|
||||
resolved[k] = trigger_values.get(src_field) # type: ignore[assignment]
|
||||
else:
|
||||
resolved[k] = v # type: ignore[assignment]
|
||||
await self._service.create_record(target_table_id, resolved)
|
||||
|
||||
async def _action_update_record(
|
||||
self,
|
||||
config: dict[str, Any],
|
||||
trigger_record_id: str | None,
|
||||
trigger_values: dict[str, object],
|
||||
) -> None:
|
||||
"""Update a record's fields (R14 update_record).
|
||||
|
||||
Config: ``{target_record_id (optional, defaults to trigger), values: {...}}``
|
||||
"""
|
||||
target_record_id = str(config.get("target_record_id") or trigger_record_id or "")
|
||||
if not target_record_id:
|
||||
raise ValueError("update_record action: no target_record_id and no trigger record")
|
||||
raw_values: dict[str, object] = config.get("values", {}) # type: ignore[assignment]
|
||||
resolved: dict[str, str | int | float | None] = {}
|
||||
for k, v in raw_values.items():
|
||||
if isinstance(v, str) and v.startswith("@trigger."):
|
||||
src_field = v[len("@trigger.") :]
|
||||
resolved[k] = trigger_values.get(src_field) # type: ignore[assignment]
|
||||
else:
|
||||
resolved[k] = v # type: ignore[assignment]
|
||||
await self._service.update_record_values(target_record_id, resolved)
|
||||
|
||||
async def _action_send_webhook(self, config: dict[str, Any], job: dict[str, Any]) -> None:
|
||||
"""POST a JSON payload to a webhook URL (R14 send_webhook).
|
||||
|
||||
Config: ``{url, headers (optional), body_template (optional)}``
|
||||
|
||||
SSRF guard (KTD-9): the URL hostname is validated against
|
||||
private/loopback/reserved IPs via ``_assert_safe_host`` before sending.
|
||||
"""
|
||||
url = str(config.get("url", ""))
|
||||
if not url:
|
||||
raise ValueError("send_webhook action missing url")
|
||||
# SSRF protection (KTD-9): reject private/loopback/reserved targets
|
||||
from urllib.parse import urlparse
|
||||
|
||||
parsed = urlparse(url)
|
||||
if parsed.scheme not in ("http", "https"):
|
||||
raise ValueError(f"Disallowed webhook URL scheme: {parsed.scheme!r}")
|
||||
if not parsed.hostname:
|
||||
raise ValueError("Webhook URL has no hostname")
|
||||
_assert_safe_host(parsed.hostname)
|
||||
|
||||
headers = config.get("headers", {"Content-Type": "application/json"})
|
||||
body_template = config.get("body_template")
|
||||
if body_template:
|
||||
# ponytail: simple @trigger.X substitution in JSON body.
|
||||
body = json.dumps(body_template).replace(
|
||||
"@trigger.record_id", str(job.get("record_id") or "")
|
||||
)
|
||||
else:
|
||||
body = json.dumps(
|
||||
{
|
||||
"event": job.get("trigger_type"),
|
||||
"table_id": job.get("table_id"),
|
||||
"record_id": job.get("record_id"),
|
||||
"values": job.get("record_values", {}),
|
||||
}
|
||||
)
|
||||
async with httpx.AsyncClient(timeout=_WEBHOOK_TIMEOUT, follow_redirects=False) as client:
|
||||
resp = await client.post(url, content=body, headers=headers)
|
||||
resp.raise_for_status()
|
||||
|
||||
def _action_send_email(self, config: dict[str, Any], job: dict[str, Any]) -> None:
|
||||
"""Send email — placeholder (SMTP integration is app-level).
|
||||
|
||||
Logs the email content; production deployments override this with
|
||||
an actual SMTP/SES/SendGrid call.
|
||||
"""
|
||||
logger.info(
|
||||
"AutomationEngine send_email (placeholder): to=%s subject=%s",
|
||||
config.get("to"),
|
||||
config.get("subject"),
|
||||
)
|
||||
|
||||
def _action_send_message(self, config: dict[str, Any], job: dict[str, Any]) -> None:
|
||||
"""Send message — placeholder (channel integration is app-level).
|
||||
|
||||
Logs the message; production deployments override this with
|
||||
Slack/DingTalk/Feishu integration.
|
||||
"""
|
||||
logger.info(
|
||||
"AutomationEngine send_message (placeholder): channel=%s text=%s",
|
||||
config.get("channel"),
|
||||
config.get("text"),
|
||||
)
|
||||
|
||||
|
||||
def generate_webhook_token() -> str:
|
||||
"""Generate a cryptographically-secure webhook token (R16).
|
||||
|
||||
Uses ``secrets.token_urlsafe(32)`` for 256 bits of entropy.
|
||||
"""
|
||||
return "whk_" + secrets.token_urlsafe(32)
|
||||
|
|
@ -21,9 +21,11 @@ from datetime import datetime, timezone
|
|||
from types import TracebackType
|
||||
|
||||
from sqlalchemy import (
|
||||
Boolean,
|
||||
Column,
|
||||
DateTime,
|
||||
Index,
|
||||
Integer,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
|
|
@ -37,7 +39,9 @@ logger = logging.getLogger(__name__)
|
|||
# Current schema version — bump when adding migrations.
|
||||
# V1: initial schema (tables/fields/records/views/recalc/meta).
|
||||
# V2: add bitable_files table + file_id column on bitable_tables (R1).
|
||||
_SCHEMA_VERSION = 2
|
||||
# V3: add relation_links / automations / automation_logs / cross_table_deps
|
||||
# tables + is_cross_table column on bitable_recalc_queue (R5/R9/R10/R16).
|
||||
_SCHEMA_VERSION = 3
|
||||
|
||||
_META_SCHEMA_VERSION_KEY = "schema_version"
|
||||
|
||||
|
|
@ -155,6 +159,7 @@ class RecalcQueueModel(BitableBase):
|
|||
|
||||
The ``(record_id, field_id)`` unique index prevents duplicate enqueues.
|
||||
The ``(status, queued_at)`` index supports efficient worker consumption.
|
||||
``is_cross_table`` marks tasks triggered by a cross-table dependency (V3).
|
||||
"""
|
||||
|
||||
__tablename__ = "bitable_recalc_queue"
|
||||
|
|
@ -172,6 +177,7 @@ class RecalcQueueModel(BitableBase):
|
|||
error_message = Column(Text, nullable=True)
|
||||
queued_at = Column(DateTime(timezone=True), default=_utcnow)
|
||||
completed_at = Column(DateTime(timezone=True), nullable=True)
|
||||
is_cross_table = Column(Boolean, default=False)
|
||||
|
||||
|
||||
class MetaModel(BitableBase):
|
||||
|
|
@ -185,6 +191,106 @@ class MetaModel(BitableBase):
|
|||
updated_at = Column(DateTime(timezone=True), default=_utcnow, onupdate=_utcnow)
|
||||
|
||||
|
||||
class RelationLinkModel(BitableBase):
|
||||
"""ORM model for ``bitable.bitable_relation_links`` — junction table (KTD-1, V3).
|
||||
|
||||
Stores many_to_many relation pairs as
|
||||
``(relation_field_id, source_record_id, target_record_id)``. Two indexes
|
||||
cover forward (source→target) and reverse (target→source) lookups for
|
||||
bidirectional relations.
|
||||
"""
|
||||
|
||||
__tablename__ = "bitable_relation_links"
|
||||
__table_args__ = (
|
||||
Index(
|
||||
"ix_relation_links_field_source",
|
||||
"relation_field_id",
|
||||
"source_record_id",
|
||||
),
|
||||
Index(
|
||||
"ix_relation_links_field_target",
|
||||
"relation_field_id",
|
||||
"target_record_id",
|
||||
),
|
||||
{"schema": "bitable"},
|
||||
)
|
||||
|
||||
id = Column(String, primary_key=True, default=_uuid_str)
|
||||
relation_field_id = Column(String, nullable=False)
|
||||
source_record_id = Column(String, nullable=False)
|
||||
target_record_id = Column(String, nullable=False)
|
||||
created_at = Column(DateTime(timezone=True), default=_utcnow)
|
||||
|
||||
|
||||
class AutomationModel(BitableBase):
|
||||
"""ORM model for ``bitable.bitable_automations`` — automation rules (V3, R16)."""
|
||||
|
||||
__tablename__ = "bitable_automations"
|
||||
__table_args__ = (
|
||||
Index("ix_automations_table_id", "table_id"),
|
||||
Index("ix_automations_webhook_token", "webhook_token"),
|
||||
{"schema": "bitable"},
|
||||
)
|
||||
|
||||
id = Column(String, primary_key=True, default=_uuid_str)
|
||||
table_id = Column(String, nullable=False)
|
||||
name = Column(String, nullable=False)
|
||||
trigger_config = Column(JSONB, default=dict)
|
||||
action_config = Column(JSONB, default=dict)
|
||||
enabled = Column(Boolean, default=True)
|
||||
webhook_token = Column(String, nullable=True)
|
||||
created_at = Column(DateTime(timezone=True), default=_utcnow)
|
||||
updated_at = Column(DateTime(timezone=True), default=_utcnow, onupdate=_utcnow)
|
||||
|
||||
|
||||
class AutomationLogModel(BitableBase):
|
||||
"""ORM model for ``bitable.bitable_automation_logs`` — execution logs (V3, R17)."""
|
||||
|
||||
__tablename__ = "bitable_automation_logs"
|
||||
__table_args__ = (
|
||||
Index("ix_automation_logs_automation_id", "automation_id", "started_at"),
|
||||
{"schema": "bitable"},
|
||||
)
|
||||
|
||||
id = Column(String, primary_key=True, default=_uuid_str)
|
||||
automation_id = Column(String, nullable=False)
|
||||
trigger_record_id = Column(String, nullable=True)
|
||||
status = Column(String, default="success")
|
||||
error_message = Column(Text, nullable=True)
|
||||
attempt = Column(Integer, default=1)
|
||||
started_at = Column(DateTime(timezone=True), default=_utcnow)
|
||||
completed_at = Column(DateTime(timezone=True), nullable=True)
|
||||
|
||||
|
||||
class CrossTableDepModel(BitableBase):
|
||||
"""ORM model for ``bitable.bitable_cross_table_deps`` — materialized dep graph (V3, KTD-3).
|
||||
|
||||
One row per ``(source_field, target_field)`` edge. ``target_field_id``
|
||||
index supports the recalc worker's reverse-edge lookup: "which fields in
|
||||
other tables depend on this changed field?"
|
||||
"""
|
||||
|
||||
__tablename__ = "bitable_cross_table_deps"
|
||||
__table_args__ = (
|
||||
UniqueConstraint(
|
||||
"source_field_id",
|
||||
"target_field_id",
|
||||
name="uq_cross_table_dep_source_target",
|
||||
),
|
||||
Index("ix_cross_table_deps_target_field", "target_field_id"),
|
||||
Index("ix_cross_table_deps_target_table", "target_table_id"),
|
||||
{"schema": "bitable"},
|
||||
)
|
||||
|
||||
id = Column(String, primary_key=True, default=_uuid_str)
|
||||
source_table_id = Column(String, nullable=False)
|
||||
source_field_id = Column(String, nullable=False)
|
||||
target_table_id = Column(String, nullable=False)
|
||||
target_field_id = Column(String, nullable=False)
|
||||
dep_type = Column(String, default="formula")
|
||||
created_at = Column(DateTime(timezone=True), default=_utcnow)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Schema is created via BitableBase.metadata.create_all (see BitableDB.init).
|
||||
# The ORM models above are the single source of truth for table/index DDL.
|
||||
|
|
@ -242,6 +348,132 @@ async def _apply_v2_migration(conn: object) -> None:
|
|||
logger.info("applied V2 migration: bitable_files table + file_id column")
|
||||
|
||||
|
||||
async def _apply_v3_migration(conn: object) -> None:
|
||||
"""V3 migration: add relation_links, automations, automation_logs,
|
||||
cross_table_deps tables + ``is_cross_table`` column on recalc_queue.
|
||||
|
||||
Idempotent — ``create_all`` already made the V3 tables on fresh installs;
|
||||
the raw SQL here ensures V2→V3 upgrades (where ``create_all`` skips
|
||||
existing tables) also get the new schema. ``ADD COLUMN IF NOT EXISTS``
|
||||
covers the recalc_queue extension.
|
||||
"""
|
||||
# 1. bitable_relation_links (junction table for many_to_many, KTD-1)
|
||||
await conn.execute(
|
||||
text(
|
||||
"CREATE TABLE IF NOT EXISTS bitable.bitable_relation_links ("
|
||||
" id VARCHAR PRIMARY KEY,"
|
||||
" relation_field_id VARCHAR NOT NULL,"
|
||||
" source_record_id VARCHAR NOT NULL,"
|
||||
" target_record_id VARCHAR NOT NULL,"
|
||||
" created_at TIMESTAMPTZ DEFAULT NOW()"
|
||||
")"
|
||||
)
|
||||
)
|
||||
await conn.execute(
|
||||
text(
|
||||
"CREATE INDEX IF NOT EXISTS ix_relation_links_field_source "
|
||||
"ON bitable.bitable_relation_links (relation_field_id, source_record_id)"
|
||||
)
|
||||
)
|
||||
await conn.execute(
|
||||
text(
|
||||
"CREATE INDEX IF NOT EXISTS ix_relation_links_field_target "
|
||||
"ON bitable.bitable_relation_links (relation_field_id, target_record_id)"
|
||||
)
|
||||
)
|
||||
|
||||
# 2. bitable_automations
|
||||
await conn.execute(
|
||||
text(
|
||||
"CREATE TABLE IF NOT EXISTS bitable.bitable_automations ("
|
||||
" id VARCHAR PRIMARY KEY,"
|
||||
" table_id VARCHAR NOT NULL,"
|
||||
" name VARCHAR NOT NULL,"
|
||||
" trigger_config JSONB DEFAULT '{}'::jsonb,"
|
||||
" action_config JSONB DEFAULT '{}'::jsonb,"
|
||||
" enabled BOOLEAN DEFAULT TRUE,"
|
||||
" webhook_token VARCHAR,"
|
||||
" created_at TIMESTAMPTZ DEFAULT NOW(),"
|
||||
" updated_at TIMESTAMPTZ DEFAULT NOW()"
|
||||
")"
|
||||
)
|
||||
)
|
||||
await conn.execute(
|
||||
text(
|
||||
"CREATE INDEX IF NOT EXISTS ix_automations_table_id "
|
||||
"ON bitable.bitable_automations (table_id)"
|
||||
)
|
||||
)
|
||||
await conn.execute(
|
||||
text(
|
||||
"CREATE INDEX IF NOT EXISTS ix_automations_webhook_token "
|
||||
"ON bitable.bitable_automations (webhook_token) "
|
||||
"WHERE webhook_token IS NOT NULL"
|
||||
)
|
||||
)
|
||||
|
||||
# 3. bitable_automation_logs
|
||||
await conn.execute(
|
||||
text(
|
||||
"CREATE TABLE IF NOT EXISTS bitable.bitable_automation_logs ("
|
||||
" id VARCHAR PRIMARY KEY,"
|
||||
" automation_id VARCHAR NOT NULL,"
|
||||
" trigger_record_id VARCHAR,"
|
||||
" status VARCHAR DEFAULT 'success',"
|
||||
" error_message TEXT,"
|
||||
" attempt INTEGER DEFAULT 1,"
|
||||
" started_at TIMESTAMPTZ DEFAULT NOW(),"
|
||||
" completed_at TIMESTAMPTZ"
|
||||
")"
|
||||
)
|
||||
)
|
||||
await conn.execute(
|
||||
text(
|
||||
"CREATE INDEX IF NOT EXISTS ix_automation_logs_automation_id "
|
||||
"ON bitable.bitable_automation_logs (automation_id, started_at)"
|
||||
)
|
||||
)
|
||||
|
||||
# 4. bitable_cross_table_deps
|
||||
await conn.execute(
|
||||
text(
|
||||
"CREATE TABLE IF NOT EXISTS bitable.bitable_cross_table_deps ("
|
||||
" id VARCHAR PRIMARY KEY,"
|
||||
" source_table_id VARCHAR NOT NULL,"
|
||||
" source_field_id VARCHAR NOT NULL,"
|
||||
" target_table_id VARCHAR NOT NULL,"
|
||||
" target_field_id VARCHAR NOT NULL,"
|
||||
" dep_type VARCHAR DEFAULT 'formula',"
|
||||
" created_at TIMESTAMPTZ DEFAULT NOW(),"
|
||||
" CONSTRAINT uq_cross_table_dep_source_target "
|
||||
" UNIQUE (source_field_id, target_field_id)"
|
||||
")"
|
||||
)
|
||||
)
|
||||
await conn.execute(
|
||||
text(
|
||||
"CREATE INDEX IF NOT EXISTS ix_cross_table_deps_target_field "
|
||||
"ON bitable.bitable_cross_table_deps (target_field_id)"
|
||||
)
|
||||
)
|
||||
await conn.execute(
|
||||
text(
|
||||
"CREATE INDEX IF NOT EXISTS ix_cross_table_deps_target_table "
|
||||
"ON bitable.bitable_cross_table_deps (target_table_id)"
|
||||
)
|
||||
)
|
||||
|
||||
# 5. is_cross_table column on recalc_queue
|
||||
await conn.execute(
|
||||
text(
|
||||
"ALTER TABLE bitable.bitable_recalc_queue "
|
||||
"ADD COLUMN IF NOT EXISTS is_cross_table BOOLEAN DEFAULT FALSE"
|
||||
)
|
||||
)
|
||||
|
||||
logger.info("applied V3 migration: relation_links / automations / logs / cross_table_deps + is_cross_table")
|
||||
|
||||
|
||||
def _resolve_database_url(database_url: str | None = None) -> str | None:
|
||||
"""Resolve PostgreSQL connection URL.
|
||||
|
||||
|
|
@ -343,6 +575,9 @@ class BitableDB:
|
|||
# V2: file layer (bitable_files table + file_id column on tables)
|
||||
if current_version < 2:
|
||||
await _apply_v2_migration(conn)
|
||||
# V3: relation_links + automations + logs + cross_table_deps + is_cross_table
|
||||
if current_version < 3:
|
||||
await _apply_v3_migration(conn)
|
||||
await conn.execute(
|
||||
text(
|
||||
"INSERT INTO bitable.bitable_meta (key, value, updated_at) "
|
||||
|
|
|
|||
|
|
@ -16,18 +16,22 @@ from agentkit.bitable.formula.engine import (
|
|||
)
|
||||
from agentkit.bitable.formula.functions import FUNCTION_REGISTRY
|
||||
from agentkit.bitable.formula.parser import (
|
||||
FormulaDepthExceededError,
|
||||
FormulaParseError,
|
||||
FormulaSecurityError,
|
||||
MAX_AST_DEPTH,
|
||||
UnknownFunctionError,
|
||||
parse_formula,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"CircularReferenceError",
|
||||
"FormulaDepthExceededError",
|
||||
"FormulaEngine",
|
||||
"FormulaParseError",
|
||||
"FormulaSecurityError",
|
||||
"FUNCTION_REGISTRY",
|
||||
"MAX_AST_DEPTH",
|
||||
"UnknownFunctionError",
|
||||
"parse_formula",
|
||||
]
|
||||
|
|
|
|||
|
|
@ -19,7 +19,10 @@ from collections import deque
|
|||
|
||||
from agentkit.bitable.formula.functions import AGGREGATE_FUNCTIONS, FUNCTION_REGISTRY
|
||||
from agentkit.bitable.formula.parser import (
|
||||
FormulaDepthExceededError,
|
||||
FormulaParseError,
|
||||
FormulaSecurityError,
|
||||
UnknownFunctionError,
|
||||
evaluate_ast,
|
||||
parse_formula,
|
||||
)
|
||||
|
|
@ -56,7 +59,9 @@ class FormulaEngine:
|
|||
UnknownFunctionError: Unregistered function.
|
||||
CircularReferenceError: Adding this formula creates a cycle.
|
||||
"""
|
||||
tree, field_mapping = parse_formula(formula, set(FUNCTION_REGISTRY.keys()))
|
||||
tree, field_mapping, cross_table_mapping = parse_formula(
|
||||
formula, set(FUNCTION_REGISTRY.keys())
|
||||
)
|
||||
|
||||
# Classify field refs into aggregate vs row context
|
||||
aggregate_refs, row_refs = _classify_refs(tree, field_mapping)
|
||||
|
|
@ -66,11 +71,14 @@ class FormulaEngine:
|
|||
field_mapping=field_mapping,
|
||||
aggregate_refs=aggregate_refs,
|
||||
row_refs=row_refs,
|
||||
cross_table_mapping=cross_table_mapping,
|
||||
formula=formula,
|
||||
)
|
||||
self._formulas[field_id] = entry
|
||||
# Update DAG: this field depends on all referenced fields
|
||||
self._dag[field_id] = aggregate_refs | row_refs
|
||||
# Cross-table refs depend on the relation field (local) + target field (foreign)
|
||||
cross_table_local_deps = {rel_id for rel_id, _ in cross_table_mapping.values()}
|
||||
self._dag[field_id] = aggregate_refs | row_refs | cross_table_local_deps
|
||||
|
||||
# Check for cycles
|
||||
cycle = _detect_cycle(self._dag)
|
||||
|
|
@ -92,6 +100,17 @@ class FormulaEngine:
|
|||
"""Get the set of field IDs that ``field_id`` depends on."""
|
||||
return self._dag.get(field_id, set()).copy()
|
||||
|
||||
def get_cross_table_mapping(self, field_id: str) -> dict[str, tuple[str, str]] | None:
|
||||
"""Get the cross-table reference mapping for a formula field.
|
||||
|
||||
Returns ``{safe_name: (relation_field_id, target_field_id)}`` or
|
||||
``None`` if the field is not registered or has no cross-table refs.
|
||||
"""
|
||||
entry = self._formulas.get(field_id)
|
||||
if entry is None:
|
||||
return None
|
||||
return entry.cross_table_mapping or None
|
||||
|
||||
def get_dependents(self, field_id: str) -> set[str]:
|
||||
"""Get the set of formula field IDs that depend on ``field_id``."""
|
||||
return {fid for fid, deps in self._dag.items() if field_id in deps}
|
||||
|
|
@ -105,6 +124,7 @@ class FormulaEngine:
|
|||
field_id: str,
|
||||
row_values: dict[str, object],
|
||||
column_values: dict[str, list[object]] | None = None,
|
||||
cross_table_values: dict[str, list[object]] | None = None,
|
||||
) -> object:
|
||||
"""Evaluate a formula field for a specific record.
|
||||
|
||||
|
|
@ -113,6 +133,9 @@ class FormulaEngine:
|
|||
row_values: Field ID → value for the current record (row context).
|
||||
column_values: Field ID → list of all values in that column
|
||||
(aggregate context). Required for aggregate references.
|
||||
cross_table_values: Safe-name → list of values from related
|
||||
records (cross-table context, U3). The service layer resolves
|
||||
these before calling evaluate.
|
||||
|
||||
Returns:
|
||||
The computed value.
|
||||
|
|
@ -126,6 +149,7 @@ class FormulaEngine:
|
|||
|
||||
entry = self._formulas[field_id]
|
||||
column_values = column_values or {}
|
||||
cross_table_values = cross_table_values or {}
|
||||
|
||||
# Build the field_values dict for the evaluator
|
||||
# Aggregate refs get column values (lists), row refs get row values (scalars)
|
||||
|
|
@ -138,6 +162,13 @@ class FormulaEngine:
|
|||
else:
|
||||
eval_values[safe_name] = row_values.get(real_id)
|
||||
|
||||
# Cross-table refs: the service layer pre-resolves these to value lists
|
||||
for safe_name in entry.cross_table_mapping:
|
||||
if safe_name in cross_table_values:
|
||||
eval_values[safe_name] = cross_table_values[safe_name]
|
||||
else:
|
||||
eval_values[safe_name] = []
|
||||
|
||||
return evaluate_ast(entry.tree, eval_values, FUNCTION_REGISTRY)
|
||||
|
||||
def evaluate_all_for_record(
|
||||
|
|
@ -150,6 +181,9 @@ class FormulaEngine:
|
|||
Returns a dict of field_id → computed value.
|
||||
Formulas are evaluated in topological order so that formula-to-formula
|
||||
dependencies are resolved correctly.
|
||||
|
||||
List return values (from FILTER/SPLIT/LOOKUP) are stored as-is.
|
||||
Downstream formulas referencing a list-valued field receive the list.
|
||||
"""
|
||||
results: dict[str, object] = {}
|
||||
column_values = column_values or {}
|
||||
|
|
@ -159,7 +193,15 @@ class FormulaEngine:
|
|||
merged_row = {**row_values, **results}
|
||||
try:
|
||||
results[field_id] = self.evaluate(field_id, merged_row, column_values)
|
||||
except (FormulaParseError, ZeroDivisionError, TypeError) as e:
|
||||
except (
|
||||
FormulaParseError,
|
||||
FormulaSecurityError,
|
||||
UnknownFunctionError,
|
||||
FormulaDepthExceededError,
|
||||
ZeroDivisionError,
|
||||
TypeError,
|
||||
ValueError,
|
||||
) as e:
|
||||
results[field_id] = {"__error": str(e)}
|
||||
|
||||
return results
|
||||
|
|
@ -171,7 +213,14 @@ class FormulaEngine:
|
|||
class _FormulaEntry:
|
||||
"""Parsed formula metadata."""
|
||||
|
||||
__slots__ = ("tree", "field_mapping", "aggregate_refs", "row_refs", "formula")
|
||||
__slots__ = (
|
||||
"tree",
|
||||
"field_mapping",
|
||||
"aggregate_refs",
|
||||
"row_refs",
|
||||
"cross_table_mapping",
|
||||
"formula",
|
||||
)
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
|
|
@ -179,12 +228,14 @@ class _FormulaEntry:
|
|||
field_mapping: dict[str, str],
|
||||
aggregate_refs: set[str],
|
||||
row_refs: set[str],
|
||||
cross_table_mapping: dict[str, tuple[str, str]],
|
||||
formula: str,
|
||||
) -> None:
|
||||
self.tree = tree
|
||||
self.field_mapping = field_mapping
|
||||
self.aggregate_refs = aggregate_refs
|
||||
self.row_refs = row_refs
|
||||
self.cross_table_mapping = cross_table_mapping
|
||||
self.formula = formula
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,110 +0,0 @@
|
|||
"""Built-in formula functions for the bitable formula engine.
|
||||
|
||||
v1 implements: SUM, AVG, COUNT, MIN, MAX, CONCAT, ABS, ROUND, IF, LEN.
|
||||
|
||||
Aggregate functions (SUM/AVG/COUNT/MIN/MAX) accept a list of values
|
||||
(the entire column) and return a scalar. Non-aggregate functions
|
||||
(ABS/ROUND/IF/LEN/CONCAT) operate on scalars.
|
||||
|
||||
The engine determines whether to pass a column (list) or scalar (row value)
|
||||
based on the calling context — see :mod:`agentkit.bitable.formula.engine`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Callable, TypeAlias
|
||||
|
||||
# A formula evaluates to a scalar primitive: text, number, or nothing.
|
||||
# bool is intentionally excluded — comparisons live in the parser layer
|
||||
# and never reach the function registry.
|
||||
FormulaResult: TypeAlias = str | int | float | None
|
||||
|
||||
# ── Aggregate functions (operate on lists) ────────────────
|
||||
|
||||
|
||||
def _sum(values: list[FormulaResult]) -> float | int:
|
||||
"""Sum of numeric values, ignoring None/empty."""
|
||||
total = 0
|
||||
for v in values:
|
||||
if v is None or v == "":
|
||||
continue
|
||||
total += v
|
||||
return total
|
||||
|
||||
|
||||
def _avg(values: list[FormulaResult]) -> float:
|
||||
"""Average of numeric values, ignoring None/empty."""
|
||||
nums = [v for v in values if v is not None and v != ""]
|
||||
if not nums:
|
||||
return 0.0
|
||||
return sum(nums) / len(nums)
|
||||
|
||||
|
||||
def _count(values: list[FormulaResult]) -> int:
|
||||
"""Count of non-empty values."""
|
||||
return sum(1 for v in values if v is not None and v != "")
|
||||
|
||||
|
||||
def _min(values: list[FormulaResult]) -> FormulaResult:
|
||||
"""Minimum of numeric values, ignoring None/empty."""
|
||||
nums = [v for v in values if v is not None and v != ""]
|
||||
if not nums:
|
||||
return 0
|
||||
return min(nums)
|
||||
|
||||
|
||||
def _max(values: list[FormulaResult]) -> FormulaResult:
|
||||
"""Maximum of numeric values, ignoring None/empty."""
|
||||
nums = [v for v in values if v is not None and v != ""]
|
||||
if not nums:
|
||||
return 0
|
||||
return max(nums)
|
||||
|
||||
|
||||
# ── Scalar functions ──────────────────────────────────────
|
||||
|
||||
|
||||
def _abs(value: FormulaResult) -> FormulaResult:
|
||||
return abs(value)
|
||||
|
||||
|
||||
def _round(value: FormulaResult, digits: int = 0) -> float:
|
||||
return round(value, digits)
|
||||
|
||||
|
||||
def _if(
|
||||
condition: FormulaResult,
|
||||
true_val: FormulaResult,
|
||||
false_val: FormulaResult = None,
|
||||
) -> FormulaResult:
|
||||
return true_val if condition else false_val
|
||||
|
||||
|
||||
def _len(value: FormulaResult) -> int:
|
||||
if value is None:
|
||||
return 0
|
||||
return len(str(value))
|
||||
|
||||
|
||||
def _concat(*args: FormulaResult) -> str:
|
||||
"""Concatenate all arguments as strings."""
|
||||
return "".join(str(a) for a in args if a is not None)
|
||||
|
||||
|
||||
# ── Registry ──────────────────────────────────────────────
|
||||
|
||||
# Functions that aggregate a column (receive a list of all column values)
|
||||
AGGREGATE_FUNCTIONS: frozenset[str] = frozenset({"SUM", "AVG", "COUNT", "MIN", "MAX"})
|
||||
|
||||
FUNCTION_REGISTRY: dict[str, Callable[..., FormulaResult]] = {
|
||||
"SUM": _sum,
|
||||
"AVG": _avg,
|
||||
"COUNT": _count,
|
||||
"MIN": _min,
|
||||
"MAX": _max,
|
||||
"ABS": _abs,
|
||||
"ROUND": _round,
|
||||
"IF": _if,
|
||||
"LEN": _len,
|
||||
"CONCAT": _concat,
|
||||
}
|
||||
|
|
@ -0,0 +1,64 @@
|
|||
"""Built-in formula function registry for the bitable formula engine.
|
||||
|
||||
This is a package (not a single module) — functions are split by category
|
||||
per KTD-6:
|
||||
- :mod:`agentkit.bitable.formula.functions.aggregate` — SUM, AVG, COUNT, MIN, MAX, ABS, ROUND, SUMIF, COUNTIF, AVERAGEIF, SUMIFS, COUNTIFS
|
||||
- :mod:`agentkit.bitable.formula.functions.datetime` — DATE, TODAY, NOW, YEAR, MONTH, DAY, HOUR, MINUTE, SECOND, WEEKDAY, DATEDIF, DATEADD, NETWORKDAYS, EOMONTH, TIMESTAMP
|
||||
- :mod:`agentkit.bitable.formula.functions.logical` — IF, IFS, SWITCH, IFERROR, COALESCE, AND, OR, NOT, ISBLANK, ISNUMBER, ISTEXT, ISDATE, TRUE, FALSE, NULL
|
||||
- :mod:`agentkit.bitable.formula.functions.text` — CONCAT, LEN, FIND, MID, LEFT, RIGHT, REPLACE, SUBSTITUTE, UPPER, LOWER, TRIM, REPT, TEXT, VALUE, STARTSWITH, ENDSWITH, SPLIT
|
||||
- :mod:`agentkit.bitable.formula.functions.lookup` — LOOKUP, FILTER, MATCH, VLOOKUP, XLOOKUP
|
||||
|
||||
Aggregate functions (SUM/AVG/COUNT/MIN/MAX and conditional variants)
|
||||
receive a list of all column values from the engine. Non-aggregate
|
||||
functions operate on scalars (row context). The engine's
|
||||
``_classify_refs()`` decides context based on AGGREGATE_FUNCTIONS.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Callable, TypeAlias
|
||||
|
||||
# A formula evaluates to a scalar primitive: text, number, bool, or nothing.
|
||||
# Lookup/FILTER may also return lists; the engine tolerates this at runtime.
|
||||
# bool is included for logical functions (TRUE/FALSE/AND/OR/NOT/predicates).
|
||||
FormulaResult: TypeAlias = str | int | float | bool | None
|
||||
|
||||
from agentkit.bitable.formula.functions.aggregate import AGGREGATE_FUNCTIONS_DICT # noqa: E402
|
||||
from agentkit.bitable.formula.functions.datetime import DATETIME_FUNCTIONS # noqa: E402
|
||||
from agentkit.bitable.formula.functions.logical import LOGICAL_FUNCTIONS # noqa: E402
|
||||
from agentkit.bitable.formula.functions.lookup import LOOKUP_FUNCTIONS # noqa: E402
|
||||
from agentkit.bitable.formula.functions.text import TEXT_FUNCTIONS # noqa: E402
|
||||
|
||||
# ── Aggregate function names (receive column lists) ──────
|
||||
#: Functions that aggregate a column (receive a list of all column values).
|
||||
#: Used by engine._classify_refs() to distinguish aggregate vs row context.
|
||||
AGGREGATE_FUNCTIONS: frozenset[str] = frozenset(
|
||||
{
|
||||
"SUM",
|
||||
"AVG",
|
||||
"COUNT",
|
||||
"MIN",
|
||||
"MAX",
|
||||
"SUMIF",
|
||||
"COUNTIF",
|
||||
"AVERAGEIF",
|
||||
"SUMIFS",
|
||||
"COUNTIFS",
|
||||
}
|
||||
)
|
||||
|
||||
# ── Unified registry ────────────────────────────────────
|
||||
#: All registered functions. Merged from category submodules.
|
||||
FUNCTION_REGISTRY: dict[str, Callable[..., FormulaResult]] = {
|
||||
**AGGREGATE_FUNCTIONS_DICT,
|
||||
**DATETIME_FUNCTIONS,
|
||||
**LOGICAL_FUNCTIONS,
|
||||
**TEXT_FUNCTIONS,
|
||||
**LOOKUP_FUNCTIONS,
|
||||
}
|
||||
|
||||
__all__ = [
|
||||
"AGGREGATE_FUNCTIONS",
|
||||
"FUNCTION_REGISTRY",
|
||||
"FormulaResult",
|
||||
]
|
||||
|
|
@ -0,0 +1,251 @@
|
|||
"""Aggregate and math formula functions (R1).
|
||||
|
||||
Aggregate functions (SUM/AVG/COUNT/MIN/MAX and conditional variants)
|
||||
operate on lists — the entire column. Scalar math functions (ABS, ROUND)
|
||||
operate on single values.
|
||||
|
||||
The engine's ``_classify_refs()`` decides whether a field reference is
|
||||
aggregate (column) or row (scalar) based on whether it appears as a
|
||||
direct argument to one of these functions.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Callable
|
||||
|
||||
from agentkit.bitable.formula.functions import FormulaResult
|
||||
|
||||
|
||||
def _is_numeric(v: FormulaResult) -> bool:
|
||||
return isinstance(v, (int, float)) and not isinstance(v, bool)
|
||||
|
||||
|
||||
def _to_number(v: FormulaResult) -> float | None:
|
||||
"""Coerce a value to number. Returns None if not coercible."""
|
||||
if v is None or v == "":
|
||||
return None
|
||||
if isinstance(v, bool):
|
||||
return 1.0 if v else 0.0
|
||||
if isinstance(v, (int, float)):
|
||||
return float(v)
|
||||
try:
|
||||
return float(v)
|
||||
except (ValueError, TypeError):
|
||||
return None
|
||||
|
||||
|
||||
# ── migrated aggregate functions ────────────────────────
|
||||
|
||||
|
||||
def _sum(values: list[FormulaResult]) -> float | int:
|
||||
"""Sum of numeric values, ignoring None/empty/non-numeric."""
|
||||
total: float | int = 0
|
||||
for v in values:
|
||||
n = _to_number(v)
|
||||
if n is not None:
|
||||
total += n
|
||||
return total
|
||||
|
||||
|
||||
def _avg(values: list[FormulaResult]) -> float:
|
||||
"""Average of numeric values, ignoring None/empty/non-numeric."""
|
||||
nums = [n for n in (_to_number(v) for v in values) if n is not None]
|
||||
if not nums:
|
||||
return 0.0
|
||||
return sum(nums) / len(nums)
|
||||
|
||||
|
||||
def _count(values: list[FormulaResult]) -> int:
|
||||
"""Count of non-empty values."""
|
||||
return sum(1 for v in values if v is not None and v != "")
|
||||
|
||||
|
||||
def _min(values: list[FormulaResult]) -> FormulaResult:
|
||||
"""Minimum of numeric values, ignoring None/empty."""
|
||||
nums = [n for n in (_to_number(v) for v in values) if n is not None]
|
||||
if not nums:
|
||||
return 0
|
||||
return min(nums)
|
||||
|
||||
|
||||
def _max(values: list[FormulaResult]) -> FormulaResult:
|
||||
"""Maximum of numeric values, ignoring None/empty."""
|
||||
nums = [n for n in (_to_number(v) for v in values) if n is not None]
|
||||
if not nums:
|
||||
return 0
|
||||
return max(nums)
|
||||
|
||||
|
||||
# ── migrated scalar math ────────────────────────────────
|
||||
|
||||
|
||||
def _abs(value: FormulaResult) -> FormulaResult:
|
||||
n = _to_number(value)
|
||||
if n is None:
|
||||
return None
|
||||
return abs(n)
|
||||
|
||||
|
||||
def _round(value: FormulaResult, digits: int = 0) -> float | None:
|
||||
n = _to_number(value)
|
||||
if n is None:
|
||||
return None
|
||||
return round(n, digits)
|
||||
|
||||
|
||||
# ── conditional aggregates ──────────────────────────────
|
||||
|
||||
|
||||
def _sumif(values: list[FormulaResult], criteria: FormulaResult) -> float | int:
|
||||
"""SUMIF(values, criteria) → sum of values matching criteria.
|
||||
|
||||
criteria may be a number (equality) or string (e.g. ">10").
|
||||
ponytail: criteria parser handles >, <, >=, <=, <>, = prefixes.
|
||||
"""
|
||||
total: float | int = 0
|
||||
op, target = _parse_criteria(criteria)
|
||||
for v in values:
|
||||
n = _to_number(v)
|
||||
if n is None:
|
||||
continue
|
||||
if _matches_criteria(n, op, target):
|
||||
total += n
|
||||
return total
|
||||
|
||||
|
||||
def _countif(values: list[FormulaResult], criteria: FormulaResult) -> int:
|
||||
"""COUNTIF(values, criteria) → count of values matching criteria."""
|
||||
count = 0
|
||||
op, target = _parse_criteria(criteria)
|
||||
for v in values:
|
||||
n = _to_number(v)
|
||||
if n is None:
|
||||
continue
|
||||
if _matches_criteria(n, op, target):
|
||||
count += 1
|
||||
return count
|
||||
|
||||
|
||||
def _averageif(values: list[FormulaResult], criteria: FormulaResult) -> float:
|
||||
"""AVERAGEIF(values, criteria) → average of values matching criteria."""
|
||||
nums: list[float] = []
|
||||
op, target = _parse_criteria(criteria)
|
||||
for v in values:
|
||||
n = _to_number(v)
|
||||
if n is None:
|
||||
continue
|
||||
if _matches_criteria(n, op, target):
|
||||
nums.append(n)
|
||||
if not nums:
|
||||
return 0.0
|
||||
return sum(nums) / len(nums)
|
||||
|
||||
|
||||
def _sumifs(
|
||||
values: list[FormulaResult], criteria_range: list[FormulaResult], criteria: FormulaResult
|
||||
) -> float | int:
|
||||
"""SUMIFS(values, criteria_range, criteria) → sum of values where criteria_range matches.
|
||||
|
||||
ponytail: single criteria range only. True SUMIFS in spreadsheets takes
|
||||
multiple (range, criteria) pairs. Ceiling: variadic version would require
|
||||
AST list-pair handling.
|
||||
"""
|
||||
if len(values) != len(criteria_range):
|
||||
raise ValueError("SUMIFS requires values and criteria_range of equal length")
|
||||
total: float | int = 0
|
||||
op, target = _parse_criteria(criteria)
|
||||
for v, c in zip(values, criteria_range):
|
||||
n = _to_number(v)
|
||||
if n is None:
|
||||
continue
|
||||
cn = _to_number(c)
|
||||
if cn is not None and _matches_criteria(cn, op, target):
|
||||
total += n
|
||||
return total
|
||||
|
||||
|
||||
def _countifs(
|
||||
values: list[FormulaResult], criteria_range: list[FormulaResult], criteria: FormulaResult
|
||||
) -> int:
|
||||
"""COUNTIFS(values, criteria_range, criteria) → count where criteria_range matches."""
|
||||
if len(values) != len(criteria_range):
|
||||
raise ValueError("COUNTIFS requires values and criteria_range of equal length")
|
||||
count = 0
|
||||
op, target = _parse_criteria(criteria)
|
||||
for v, c in zip(values, criteria_range):
|
||||
n = _to_number(v)
|
||||
if n is None:
|
||||
continue
|
||||
cn = _to_number(c)
|
||||
if cn is not None and _matches_criteria(cn, op, target):
|
||||
count += 1
|
||||
return count
|
||||
|
||||
|
||||
# ── criteria helpers ────────────────────────────────────
|
||||
|
||||
|
||||
def _parse_criteria(criteria: FormulaResult) -> tuple[str, float | None]:
|
||||
"""Parse a criteria value into (operator, target_number).
|
||||
|
||||
Supports: ">10", "<10", ">=10", "<=10", "<>10", "=10", or plain 10.
|
||||
Returns ("=", None) for non-string criteria (equality against the value itself).
|
||||
"""
|
||||
if isinstance(criteria, str):
|
||||
s = criteria.strip()
|
||||
for op_prefix in (">=", "<=", "<>", ">", "<", "="):
|
||||
if s.startswith(op_prefix):
|
||||
target_str = s[len(op_prefix) :].strip()
|
||||
try:
|
||||
return op_prefix, float(target_str)
|
||||
except ValueError:
|
||||
return op_prefix, None
|
||||
# String criteria without operator — try numeric equality
|
||||
try:
|
||||
return "=", float(s)
|
||||
except ValueError:
|
||||
return "=", None
|
||||
if isinstance(criteria, (int, float)) and not isinstance(criteria, bool):
|
||||
return "=", float(criteria)
|
||||
return "=", None
|
||||
|
||||
|
||||
def _matches_criteria(value: float, op: str, target: float | None) -> bool:
|
||||
"""Check if value matches the parsed criteria."""
|
||||
if target is None:
|
||||
return False
|
||||
if op == ">":
|
||||
return value > target
|
||||
if op == "<":
|
||||
return value < target
|
||||
if op == ">=":
|
||||
return value >= target
|
||||
if op == "<=":
|
||||
return value <= target
|
||||
if op == "<>":
|
||||
return value != target
|
||||
if op == "=":
|
||||
return value == target
|
||||
return False
|
||||
|
||||
|
||||
# ── Registry ────────────────────────────────────────────
|
||||
|
||||
AGGREGATE_FUNCTIONS_LIST: frozenset[str] = frozenset(
|
||||
{"SUM", "AVG", "COUNT", "MIN", "MAX", "SUMIF", "COUNTIF", "AVERAGEIF", "SUMIFS", "COUNTIFS"}
|
||||
)
|
||||
|
||||
AGGREGATE_FUNCTIONS_DICT: dict[str, Callable[..., FormulaResult]] = {
|
||||
"SUM": _sum,
|
||||
"AVG": _avg,
|
||||
"COUNT": _count,
|
||||
"MIN": _min,
|
||||
"MAX": _max,
|
||||
"ABS": _abs,
|
||||
"ROUND": _round,
|
||||
"SUMIF": _sumif,
|
||||
"COUNTIF": _countif,
|
||||
"AVERAGEIF": _averageif,
|
||||
"SUMIFS": _sumifs,
|
||||
"COUNTIFS": _countifs,
|
||||
}
|
||||
|
|
@ -0,0 +1,214 @@
|
|||
"""Date/time formula functions (R1).
|
||||
|
||||
All functions accept ISO 8601 strings (the storage format for ``date`` fields)
|
||||
or Python datetime objects. NETWORKDAYS uses Python's ``datetime.weekday()``
|
||||
to exclude Saturdays/Sundays; holiday calendars are out of scope for v2
|
||||
(ponytail: ceiling — upgrade path is a holidays JSON config).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime, timedelta, timezone
|
||||
from typing import Callable
|
||||
|
||||
from agentkit.bitable.formula.functions import FormulaResult
|
||||
|
||||
|
||||
def _parse_date(value: FormulaResult) -> datetime | None:
|
||||
"""Parse an ISO 8601 string or pass through a datetime.
|
||||
|
||||
Returns None if value is None/empty/invalid. Raises ValueError on
|
||||
non-date strings so callers can surface a formula error.
|
||||
"""
|
||||
if value is None or value == "":
|
||||
return None
|
||||
if isinstance(value, datetime):
|
||||
return value
|
||||
if isinstance(value, date):
|
||||
return datetime(value.year, value.month, value.day)
|
||||
if isinstance(value, str):
|
||||
# ISO 8601 — datetime.fromisoformat handles "2026-07-06" and
|
||||
# "2026-07-06T10:30:00" and "2026-07-06T10:30:00+00:00".
|
||||
return datetime.fromisoformat(value)
|
||||
# Numbers aren't dates here — surface as a formula error.
|
||||
raise ValueError(f"Cannot parse date from: {value!r}")
|
||||
|
||||
|
||||
def _to_serializable(dt: datetime) -> str:
|
||||
"""Serialize a datetime to ISO 8601 string (storage format)."""
|
||||
return dt.isoformat()
|
||||
|
||||
|
||||
# ── Constructors ─────────────────────────────────────────
|
||||
|
||||
|
||||
def _date(year: int, month: int, day: int) -> str:
|
||||
"""DATE(year, month, day) → ISO 8601 date string."""
|
||||
return _to_serializable(datetime(year, month, day))
|
||||
|
||||
|
||||
def _today() -> str:
|
||||
"""TODAY() → current date (UTC, midnight)."""
|
||||
now = datetime.now(timezone.utc)
|
||||
return _to_serializable(now.replace(hour=0, minute=0, second=0, microsecond=0))
|
||||
|
||||
|
||||
def _now() -> str:
|
||||
"""NOW() → current datetime (UTC)."""
|
||||
return _to_serializable(datetime.now(timezone.utc))
|
||||
|
||||
|
||||
def _timestamp() -> str:
|
||||
"""TIMESTAMP() → current datetime (UTC). Alias for NOW."""
|
||||
return _now()
|
||||
|
||||
|
||||
# ── Accessors ────────────────────────────────────────────
|
||||
|
||||
|
||||
def _year(value: FormulaResult) -> int | None:
|
||||
dt = _parse_date(value)
|
||||
return dt.year if dt else None
|
||||
|
||||
|
||||
def _month(value: FormulaResult) -> int | None:
|
||||
dt = _parse_date(value)
|
||||
return dt.month if dt else None
|
||||
|
||||
|
||||
def _day(value: FormulaResult) -> int | None:
|
||||
dt = _parse_date(value)
|
||||
return dt.day if dt else None
|
||||
|
||||
|
||||
def _hour(value: FormulaResult) -> int | None:
|
||||
dt = _parse_date(value)
|
||||
return dt.hour if dt else None
|
||||
|
||||
|
||||
def _minute(value: FormulaResult) -> int | None:
|
||||
dt = _parse_date(value)
|
||||
return dt.minute if dt else None
|
||||
|
||||
|
||||
def _second(value: FormulaResult) -> int | None:
|
||||
dt = _parse_date(value)
|
||||
return dt.second if dt else None
|
||||
|
||||
|
||||
def _weekday(value: FormulaResult) -> int | None:
|
||||
"""WEEKDAY(date) → day of week (1=Monday … 7=Sunday, ISO 8601)."""
|
||||
dt = _parse_date(value)
|
||||
if dt is None:
|
||||
return None
|
||||
return dt.isoweekday()
|
||||
|
||||
|
||||
# ── Arithmetic ───────────────────────────────────────────
|
||||
|
||||
|
||||
def _datedif(start: FormulaResult, end: FormulaResult, unit: str = "days") -> int | float | None:
|
||||
"""DATEDIF(start, end, unit) → difference in days/months/years.
|
||||
|
||||
unit: "days" (default), "months", "years".
|
||||
"""
|
||||
s = _parse_date(start)
|
||||
e = _parse_date(end)
|
||||
if s is None or e is None:
|
||||
return None
|
||||
if unit == "days":
|
||||
return (e - s).days
|
||||
if unit == "months":
|
||||
return (e.year - s.year) * 12 + (e.month - s.month)
|
||||
if unit == "years":
|
||||
return e.year - s.year
|
||||
raise ValueError(f"DATEDIF unit must be days/months/years, got: {unit!r}")
|
||||
|
||||
|
||||
def _dateadd(value: FormulaResult, amount: int, unit: str = "days") -> str | None:
|
||||
"""DATEADD(date, amount, unit) → new date string.
|
||||
|
||||
unit: "days" (default), "months", "years".
|
||||
"""
|
||||
dt = _parse_date(value)
|
||||
if dt is None:
|
||||
return None
|
||||
if unit == "days":
|
||||
return _to_serializable(dt + timedelta(days=amount))
|
||||
if unit == "months":
|
||||
# ponytail: naive month arithmetic — no day-of-month overflow handling.
|
||||
# Ceiling: Jan 31 + 1 month → Feb 28/29 not preserved. Upgrade path:
|
||||
# use dateutil.relativedelta (would add a dependency).
|
||||
new_month = dt.month + amount
|
||||
new_year = dt.year + (new_month - 1) // 12
|
||||
new_month = ((new_month - 1) % 12) + 1
|
||||
# Clamp day to last day of new month
|
||||
import calendar
|
||||
|
||||
last_day = calendar.monthrange(new_year, new_month)[1]
|
||||
new_day = min(dt.day, last_day)
|
||||
return _to_serializable(dt.replace(year=new_year, month=new_month, day=new_day))
|
||||
if unit == "years":
|
||||
try:
|
||||
return _to_serializable(dt.replace(year=dt.year + amount))
|
||||
except ValueError:
|
||||
# Feb 29 → non-leap year: clamp to Feb 28
|
||||
return _to_serializable(dt.replace(year=dt.year + amount, day=28))
|
||||
raise ValueError(f"DATEADD unit must be days/months/years, got: {unit!r}")
|
||||
|
||||
|
||||
def _networkdays(start: FormulaResult, end: FormulaResult) -> int | None:
|
||||
"""NETWORKDAYS(start, end) → number of working days (Mon-Fri) between dates.
|
||||
|
||||
Both endpoints inclusive. Holiday calendars are out of scope (v3+).
|
||||
"""
|
||||
s = _parse_date(start)
|
||||
e = _parse_date(end)
|
||||
if s is None or e is None:
|
||||
return None
|
||||
if s > e:
|
||||
s, e = e, s
|
||||
days = 0
|
||||
cur = s.date()
|
||||
end_date = e.date()
|
||||
while cur <= end_date:
|
||||
if cur.weekday() < 5: # Mon-Fri
|
||||
days += 1
|
||||
cur += timedelta(days=1)
|
||||
return days
|
||||
|
||||
|
||||
def _eomonth(value: FormulaResult, months: int = 0) -> str | None:
|
||||
"""EOMONTH(date, months) → last day of the month N months after date."""
|
||||
dt = _parse_date(value)
|
||||
if dt is None:
|
||||
return None
|
||||
import calendar
|
||||
|
||||
new_month = dt.month + months
|
||||
new_year = dt.year + (new_month - 1) // 12
|
||||
new_month = ((new_month - 1) % 12) + 1
|
||||
last_day = calendar.monthrange(new_year, new_month)[1]
|
||||
return _to_serializable(dt.replace(year=new_year, month=new_month, day=last_day))
|
||||
|
||||
|
||||
# ── Registry ─────────────────────────────────────────────
|
||||
|
||||
#: Functions in this module that should be registered globally.
|
||||
DATETIME_FUNCTIONS: dict[str, Callable[..., FormulaResult]] = {
|
||||
"DATE": _date,
|
||||
"DATEDIF": _datedif,
|
||||
"NETWORKDAYS": _networkdays,
|
||||
"WEEKDAY": _weekday,
|
||||
"EOMONTH": _eomonth,
|
||||
"TODAY": _today,
|
||||
"NOW": _now,
|
||||
"YEAR": _year,
|
||||
"MONTH": _month,
|
||||
"DAY": _day,
|
||||
"HOUR": _hour,
|
||||
"MINUTE": _minute,
|
||||
"SECOND": _second,
|
||||
"DATEADD": _dateadd,
|
||||
"TIMESTAMP": _timestamp,
|
||||
}
|
||||
|
|
@ -0,0 +1,164 @@
|
|||
"""Logical formula functions (R1).
|
||||
|
||||
Boolean, conditional, and type-checking functions. All operate on
|
||||
scalars (row context). Boolean returns are Python ``bool`` which the
|
||||
engine treats as truthy/falsy in conditional context.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime
|
||||
from typing import Callable
|
||||
|
||||
from agentkit.bitable.formula.functions import FormulaResult
|
||||
|
||||
|
||||
def _is_truthy(value: FormulaResult) -> bool:
|
||||
"""Truthiness for formula context: None/0/'' are falsy, everything else truthy."""
|
||||
if value is None:
|
||||
return False
|
||||
if isinstance(value, str):
|
||||
return value != "" and value.lower() != "false"
|
||||
if isinstance(value, (int, float)):
|
||||
return value != 0
|
||||
return bool(value)
|
||||
|
||||
|
||||
# ── migrated from functions.py ──────────────────────────
|
||||
|
||||
|
||||
def _if(
|
||||
condition: FormulaResult,
|
||||
true_val: FormulaResult,
|
||||
false_val: FormulaResult = None,
|
||||
) -> FormulaResult:
|
||||
return true_val if _is_truthy(condition) else false_val
|
||||
|
||||
|
||||
# ── multi-branch conditionals ───────────────────────────
|
||||
|
||||
|
||||
def _ifs(*pairs: FormulaResult) -> FormulaResult:
|
||||
"""IFS(c1, v1, c2, v2, ...) → value of first truthy condition.
|
||||
|
||||
Pairs must be (condition, value, condition, value, ...).
|
||||
Returns None if no condition matches.
|
||||
"""
|
||||
if len(pairs) % 2 != 0:
|
||||
raise ValueError("IFS requires pairs of (condition, value)")
|
||||
for i in range(0, len(pairs), 2):
|
||||
if _is_truthy(pairs[i]):
|
||||
return pairs[i + 1]
|
||||
return None
|
||||
|
||||
|
||||
def _switch(value: FormulaResult, *pairs: FormulaResult) -> FormulaResult:
|
||||
"""SWITCH(value, c1, v1, c2, v2, ..., default?) → first matching value.
|
||||
|
||||
Compares value against each case cN; returns vN on first match.
|
||||
Optional trailing default (odd arg count) returned if no match.
|
||||
"""
|
||||
n = len(pairs)
|
||||
has_default = n % 2 == 1
|
||||
limit = n - 1 if has_default else n
|
||||
for i in range(0, limit, 2):
|
||||
if value == pairs[i]:
|
||||
return pairs[i + 1]
|
||||
return pairs[-1] if has_default else None
|
||||
|
||||
|
||||
def _iferror(value: FormulaResult, default: FormulaResult = None) -> FormulaResult:
|
||||
"""IFERROR(value, default=None) → value, or default if value is an error dict.
|
||||
|
||||
The engine wraps exceptions as ``{"__error": "..."}`` dicts; this function
|
||||
unwraps them. Non-error values pass through unchanged.
|
||||
"""
|
||||
if isinstance(value, dict) and "__error" in value:
|
||||
return default
|
||||
return value
|
||||
|
||||
|
||||
def _coalesce(*values: FormulaResult) -> FormulaResult:
|
||||
"""COALESCE(v1, v2, ...) → first non-None/non-empty value."""
|
||||
for v in values:
|
||||
if v is not None and v != "":
|
||||
return v
|
||||
return None
|
||||
|
||||
|
||||
# ── boolean operators ───────────────────────────────────
|
||||
|
||||
|
||||
def _and(*args: FormulaResult) -> bool:
|
||||
return all(_is_truthy(a) for a in args)
|
||||
|
||||
|
||||
def _or(*args: FormulaResult) -> bool:
|
||||
return any(_is_truthy(a) for a in args)
|
||||
|
||||
|
||||
def _not(value: FormulaResult) -> bool:
|
||||
return not _is_truthy(value)
|
||||
|
||||
|
||||
# ── predicates ──────────────────────────────────────────
|
||||
|
||||
|
||||
def _isblank(value: FormulaResult) -> bool:
|
||||
return value is None or value == ""
|
||||
|
||||
|
||||
def _isnumber(value: FormulaResult) -> bool:
|
||||
return isinstance(value, (int, float)) and not isinstance(value, bool)
|
||||
|
||||
|
||||
def _istext(value: FormulaResult) -> bool:
|
||||
return isinstance(value, str)
|
||||
|
||||
|
||||
def _isdate(value: FormulaResult) -> bool:
|
||||
if isinstance(value, (datetime, date)):
|
||||
return True
|
||||
if isinstance(value, str):
|
||||
try:
|
||||
datetime.fromisoformat(value)
|
||||
return True
|
||||
except ValueError:
|
||||
return False
|
||||
return False
|
||||
|
||||
|
||||
# ── constants ───────────────────────────────────────────
|
||||
|
||||
|
||||
def _true() -> bool:
|
||||
return True
|
||||
|
||||
|
||||
def _false() -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def _null() -> None:
|
||||
return None
|
||||
|
||||
|
||||
# ── Registry ────────────────────────────────────────────
|
||||
|
||||
LOGICAL_FUNCTIONS: dict[str, Callable[..., FormulaResult]] = {
|
||||
"IF": _if,
|
||||
"IFS": _ifs,
|
||||
"SWITCH": _switch,
|
||||
"IFERROR": _iferror,
|
||||
"COALESCE": _coalesce,
|
||||
"AND": _and,
|
||||
"OR": _or,
|
||||
"NOT": _not,
|
||||
"ISBLANK": _isblank,
|
||||
"ISNUMBER": _isnumber,
|
||||
"ISTEXT": _istext,
|
||||
"ISDATE": _isdate,
|
||||
"TRUE": _true,
|
||||
"FALSE": _false,
|
||||
"NULL": _null,
|
||||
}
|
||||
|
|
@ -0,0 +1,189 @@
|
|||
"""Lookup formula functions (R3 partial).
|
||||
|
||||
LOOKUP / FILTER / MATCH / VLOOKUP / XLOOKUP operate on lists of values
|
||||
and return scalars or lists. Cross-table resolution (resolving relation
|
||||
fields to foreign-table columns) is deferred to U3 — these functions
|
||||
expect the engine to have already expanded relation context into flat
|
||||
value lists.
|
||||
|
||||
ponytail: ceiling — true cross-table LOOKUP requires the U3 relation
|
||||
context expansion in the engine. The functions here operate on
|
||||
in-memory lists passed by the engine, keeping them usable for same-table
|
||||
lookups now and cross-table later without API change.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Callable
|
||||
|
||||
from agentkit.bitable.formula.functions import FormulaResult
|
||||
|
||||
|
||||
def _to_number(v: FormulaResult) -> float | None:
|
||||
if v is None or v == "":
|
||||
return None
|
||||
if isinstance(v, bool):
|
||||
return 1.0 if v else 0.0
|
||||
if isinstance(v, (int, float)):
|
||||
return float(v)
|
||||
try:
|
||||
return float(v)
|
||||
except (ValueError, TypeError):
|
||||
return None
|
||||
|
||||
|
||||
def _lookup(
|
||||
lookup_value: FormulaResult,
|
||||
lookup_vector: list[FormulaResult],
|
||||
result_vector: list[FormulaResult],
|
||||
) -> FormulaResult:
|
||||
"""LOOKUP(value, lookup_vector, result_vector) → first result where lookup_vector matches.
|
||||
|
||||
Approximate match: lookup_vector must be sorted ascending. Returns the
|
||||
last result where lookup_vector value <= lookup_value.
|
||||
"""
|
||||
target = _to_number(lookup_value)
|
||||
if target is None:
|
||||
return None
|
||||
if len(lookup_vector) != len(result_vector):
|
||||
raise ValueError("LOOKUP requires lookup_vector and result_vector of equal length")
|
||||
last_match: FormulaResult = None
|
||||
for lv, rv in zip(lookup_vector, result_vector):
|
||||
n = _to_number(lv)
|
||||
if n is None:
|
||||
continue
|
||||
if n <= target:
|
||||
last_match = rv
|
||||
else:
|
||||
break
|
||||
return last_match
|
||||
|
||||
|
||||
def _filter(
|
||||
values: list[FormulaResult],
|
||||
criteria_range: list[FormulaResult],
|
||||
criteria: FormulaResult,
|
||||
) -> list[FormulaResult]:
|
||||
"""FILTER(values, criteria_range, criteria) → values where criteria_range matches.
|
||||
|
||||
Returns a list (multi-value result). The engine treats list returns as
|
||||
multi-value results — useful for downstream LOOKUP/MATCH.
|
||||
"""
|
||||
if len(values) != len(criteria_range):
|
||||
raise ValueError("FILTER requires values and criteria_range of equal length")
|
||||
target = criteria
|
||||
result: list[FormulaResult] = []
|
||||
for v, c in zip(values, criteria_range):
|
||||
if c == target:
|
||||
result.append(v)
|
||||
return result
|
||||
|
||||
|
||||
def _match(
|
||||
lookup_value: FormulaResult,
|
||||
lookup_array: list[FormulaResult],
|
||||
match_type: int = 0,
|
||||
) -> int | None:
|
||||
"""MATCH(value, lookup_array, match_type=0) → 0-based position.
|
||||
|
||||
match_type:
|
||||
0 (default) — exact match
|
||||
1 — largest value <= lookup_value (array must be sorted ascending)
|
||||
-1 — smallest value >= lookup_value (array must be sorted descending)
|
||||
Returns None if no match.
|
||||
"""
|
||||
if match_type == 0:
|
||||
for i, v in enumerate(lookup_array):
|
||||
if v == lookup_value:
|
||||
return i
|
||||
return None
|
||||
target = _to_number(lookup_value)
|
||||
if target is None:
|
||||
return None
|
||||
if match_type == 1:
|
||||
last_match: int | None = None
|
||||
for i, v in enumerate(lookup_array):
|
||||
n = _to_number(v)
|
||||
if n is None:
|
||||
continue
|
||||
if n <= target:
|
||||
last_match = i
|
||||
else:
|
||||
break
|
||||
return last_match
|
||||
if match_type == -1:
|
||||
last_match = None
|
||||
for i, v in enumerate(lookup_array):
|
||||
n = _to_number(v)
|
||||
if n is None:
|
||||
continue
|
||||
if n >= target:
|
||||
last_match = i
|
||||
else:
|
||||
break
|
||||
return last_match
|
||||
return None
|
||||
|
||||
|
||||
def _vlookup(
|
||||
lookup_value: FormulaResult,
|
||||
table_vector: list[list[FormulaResult]],
|
||||
col_index: int,
|
||||
range_lookup: bool = True,
|
||||
) -> FormulaResult:
|
||||
"""VLOOKUP(value, table, col_index, range_lookup=True) → value from col_index of matching row.
|
||||
|
||||
table_vector is a list of rows (each row a list of values).
|
||||
col_index is 1-based.
|
||||
range_lookup: True (default) = approximate match (sorted); False = exact.
|
||||
"""
|
||||
if col_index < 1:
|
||||
raise ValueError("VLOOKUP col_index must be >= 1")
|
||||
target = _to_number(lookup_value) if range_lookup else lookup_value
|
||||
last_match: FormulaResult = None
|
||||
for row in table_vector:
|
||||
if not row:
|
||||
continue
|
||||
first = row[0]
|
||||
if range_lookup:
|
||||
n = _to_number(first)
|
||||
if n is None:
|
||||
continue
|
||||
if n <= (target or 0):
|
||||
if col_index <= len(row):
|
||||
last_match = row[col_index - 1]
|
||||
else:
|
||||
break
|
||||
else:
|
||||
if first == lookup_value:
|
||||
if col_index <= len(row):
|
||||
return row[col_index - 1]
|
||||
return None
|
||||
return last_match
|
||||
|
||||
|
||||
def _xlookup(
|
||||
lookup_value: FormulaResult,
|
||||
lookup_array: list[FormulaResult],
|
||||
return_array: list[FormulaResult],
|
||||
if_not_found: FormulaResult = None,
|
||||
) -> FormulaResult:
|
||||
"""XLOOKUP(value, lookup_array, return_array, if_not_found=None) → first match's return.
|
||||
|
||||
Exact match by default. Returns if_not_found (default None) if no match.
|
||||
"""
|
||||
for lv, rv in zip(lookup_array, return_array):
|
||||
if lv == lookup_value:
|
||||
return rv
|
||||
return if_not_found
|
||||
|
||||
|
||||
# ── Registry ────────────────────────────────────────────
|
||||
|
||||
LOOKUP_FUNCTIONS: dict[str, Callable[..., FormulaResult]] = {
|
||||
"LOOKUP": _lookup,
|
||||
"FILTER": _filter,
|
||||
"MATCH": _match,
|
||||
"VLOOKUP": _vlookup,
|
||||
"XLOOKUP": _xlookup,
|
||||
}
|
||||
|
|
@ -0,0 +1,212 @@
|
|||
"""Text formula functions (R1).
|
||||
|
||||
All functions operate on scalars (row context). Storage format for text
|
||||
fields is ``str``; numbers are coerced via ``str()`` where appropriate.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Callable
|
||||
|
||||
from agentkit.bitable.formula.functions import FormulaResult
|
||||
|
||||
|
||||
def _to_str(value: FormulaResult) -> str:
|
||||
"""Coerce a formula value to string. None → empty string."""
|
||||
if value is None:
|
||||
return ""
|
||||
return str(value)
|
||||
|
||||
|
||||
# ── migrated from functions.py ──────────────────────────
|
||||
|
||||
|
||||
def _concat(*args: FormulaResult) -> str:
|
||||
"""Concatenate all arguments as strings, skipping None."""
|
||||
return "".join(_to_str(a) for a in args if a is not None)
|
||||
|
||||
|
||||
def _len(value: FormulaResult) -> int:
|
||||
"""Length of value's string form. None → 0."""
|
||||
if value is None:
|
||||
return 0
|
||||
return len(str(value))
|
||||
|
||||
|
||||
# ── find / extract ──────────────────────────────────────
|
||||
|
||||
|
||||
def _find(text: FormulaResult, substring: FormulaResult, start: int = 0) -> int:
|
||||
"""FIND(text, substring, start=0) → 0-based index, -1 if not found."""
|
||||
t = _to_str(text)
|
||||
s = _to_str(substring)
|
||||
if not s:
|
||||
return 0
|
||||
return t.find(s, max(0, start))
|
||||
|
||||
|
||||
def _mid(text: FormulaResult, start: int, length: int) -> str:
|
||||
"""MID(text, start, length) → substring from 0-based start for length chars."""
|
||||
t = _to_str(text)
|
||||
if start < 0:
|
||||
start = 0
|
||||
if length < 0:
|
||||
length = 0
|
||||
return t[start : start + length]
|
||||
|
||||
|
||||
def _left(text: FormulaResult, n: int = 1) -> str:
|
||||
"""LEFT(text, n=1) → first n characters."""
|
||||
t = _to_str(text)
|
||||
if n <= 0:
|
||||
return ""
|
||||
return t[:n]
|
||||
|
||||
|
||||
def _right(text: FormulaResult, n: int = 1) -> str:
|
||||
"""RIGHT(text, n=1) → last n characters."""
|
||||
t = _to_str(text)
|
||||
if n <= 0:
|
||||
return ""
|
||||
return t[-n:] if n <= len(t) else t
|
||||
|
||||
|
||||
# ── replace / substitute ────────────────────────────────
|
||||
|
||||
|
||||
def _replace(text: FormulaResult, start: int, length: int, new: FormulaResult) -> str:
|
||||
"""REPLACE(text, start, length, new) → text with [start:start+length] replaced by new."""
|
||||
t = _to_str(text)
|
||||
n = _to_str(new)
|
||||
if start < 0:
|
||||
start = 0
|
||||
if length < 0:
|
||||
length = 0
|
||||
return t[:start] + n + t[start + length :]
|
||||
|
||||
|
||||
def _substitute(
|
||||
text: FormulaResult, old: FormulaResult, new: FormulaResult, occurrence: int = -1
|
||||
) -> str:
|
||||
"""SUBSTITUTE(text, old, new, occurrence=-1) → text with old replaced by new.
|
||||
|
||||
occurrence: -1 (default) replaces all; otherwise replaces the Nth occurrence (1-based).
|
||||
"""
|
||||
t = _to_str(text)
|
||||
o = _to_str(old)
|
||||
n = _to_str(new)
|
||||
if not o:
|
||||
return t
|
||||
if occurrence < 0:
|
||||
return t.replace(o, n)
|
||||
# Replace only the Nth occurrence (1-based)
|
||||
parts = t.split(o)
|
||||
if len(parts) <= occurrence:
|
||||
return t # not enough occurrences
|
||||
return o.join(parts[:occurrence]) + n + o.join(parts[occurrence:])
|
||||
|
||||
|
||||
# ── case / trim ─────────────────────────────────────────
|
||||
|
||||
|
||||
def _upper(value: FormulaResult) -> str:
|
||||
return _to_str(value).upper()
|
||||
|
||||
|
||||
def _lower(value: FormulaResult) -> str:
|
||||
return _to_str(value).lower()
|
||||
|
||||
|
||||
def _trim(value: FormulaResult) -> str:
|
||||
"""TRIM(text) → strip leading/trailing whitespace and collapse internal runs."""
|
||||
return " ".join(_to_str(value).split())
|
||||
|
||||
|
||||
# ── repeat / format / convert ───────────────────────────
|
||||
|
||||
|
||||
def _rept(text: FormulaResult, n: int) -> str:
|
||||
"""REPT(text, n) → text repeated n times."""
|
||||
if n <= 0:
|
||||
return ""
|
||||
return _to_str(text) * n
|
||||
|
||||
|
||||
def _text(value: FormulaResult, fmt: str = "") -> str:
|
||||
"""TEXT(value, fmt) → value formatted as string.
|
||||
|
||||
fmt is a simple format spec passed to Python's ``format()``.
|
||||
Empty fmt → ``str(value)``.
|
||||
"""
|
||||
if value is None:
|
||||
return ""
|
||||
if not fmt:
|
||||
return str(value)
|
||||
try:
|
||||
return format(value, fmt)
|
||||
except (ValueError, TypeError):
|
||||
return str(value)
|
||||
|
||||
|
||||
def _value(text: FormulaResult) -> float | int | None:
|
||||
"""VALUE(text) → parse text as number. Returns None if unparseable."""
|
||||
if text is None:
|
||||
return None
|
||||
if isinstance(text, (int, float)):
|
||||
return text
|
||||
s = str(text).strip()
|
||||
if not s:
|
||||
return None
|
||||
try:
|
||||
# Prefer int if no decimal point
|
||||
if "." not in s and "e" not in s.lower():
|
||||
return int(s)
|
||||
return float(s)
|
||||
except ValueError:
|
||||
try:
|
||||
return float(s)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
# ── predicates / split ──────────────────────────────────
|
||||
|
||||
|
||||
def _startswith(text: FormulaResult, prefix: FormulaResult) -> bool:
|
||||
return _to_str(text).startswith(_to_str(prefix))
|
||||
|
||||
|
||||
def _endswith(text: FormulaResult, suffix: FormulaResult) -> bool:
|
||||
return _to_str(text).endswith(_to_str(suffix))
|
||||
|
||||
|
||||
def _split(text: FormulaResult, delimiter: str = ",") -> list[str]:
|
||||
"""SPLIT(text, delimiter=',') → list of substrings.
|
||||
|
||||
Note: returns a list. The formula engine treats list returns as
|
||||
multi-value results (used by LOOKUP/FILTER context in U3).
|
||||
"""
|
||||
return _to_str(text).split(delimiter) if delimiter else [_to_str(text)]
|
||||
|
||||
|
||||
# ── Registry ────────────────────────────────────────────
|
||||
|
||||
TEXT_FUNCTIONS: dict[str, Callable[..., FormulaResult]] = {
|
||||
"CONCAT": _concat,
|
||||
"LEN": _len,
|
||||
"FIND": _find,
|
||||
"MID": _mid,
|
||||
"LEFT": _left,
|
||||
"RIGHT": _right,
|
||||
"REPLACE": _replace,
|
||||
"SUBSTITUTE": _substitute,
|
||||
"UPPER": _upper,
|
||||
"LOWER": _lower,
|
||||
"TRIM": _trim,
|
||||
"REPT": _rept,
|
||||
"TEXT": _text,
|
||||
"VALUE": _value,
|
||||
"STARTSWITH": _startswith,
|
||||
"ENDSWITH": _endswith,
|
||||
"SPLIT": _split,
|
||||
}
|
||||
|
|
@ -44,30 +44,65 @@ class UnknownFunctionError(Exception):
|
|||
"""Raised when a formula calls a function not in the registry."""
|
||||
|
||||
|
||||
class FormulaDepthExceededError(Exception):
|
||||
"""Raised when a formula AST exceeds the max depth (KTD8).
|
||||
|
||||
Limits deeply nested expressions that could cause exponential
|
||||
evaluation time. Default limit: 15 levels.
|
||||
"""
|
||||
|
||||
|
||||
# KTD8: AST depth limit — prevents exponential-time evaluation of
|
||||
# deeply nested expressions (e.g. nested IF chains, arithmetic towers).
|
||||
MAX_AST_DEPTH: int = 15
|
||||
|
||||
|
||||
# ── Field reference substitution ──────────────────────────
|
||||
|
||||
# Match {field_id} — field IDs are UUIDs or alphanumeric.
|
||||
_FIELD_REF_RE = re.compile(r"\{([a-zA-Z0-9_-]+)\}")
|
||||
|
||||
# Match {rel_field_id.target_field_id} — cross-table reference (KTD-2, U3).
|
||||
# Must be matched BEFORE _FIELD_REF_RE to avoid stripping the rel part.
|
||||
_CROSS_TABLE_REF_RE = re.compile(r"\{([a-zA-Z0-9_-]+)\.([a-zA-Z0-9_-]+)\}")
|
||||
|
||||
def _substitute_field_refs(formula: str) -> tuple[str, dict[str, str]]:
|
||||
"""Replace ``{field_id}`` with ``_f_<safe_name>`` (a Python Name node).
|
||||
|
||||
Field IDs are UUIDs that may start with a digit, which is invalid in Python
|
||||
identifiers. We prefix with ``_f_`` and replace hyphens with underscores.
|
||||
A reverse mapping is returned so the engine can map back to real field IDs.
|
||||
def _substitute_field_refs(
|
||||
formula: str,
|
||||
) -> tuple[str, dict[str, str], dict[str, tuple[str, str]]]:
|
||||
"""Replace field references with safe Python identifiers.
|
||||
|
||||
Handles two syntaxes:
|
||||
- ``{field_id}`` → ``_f_<safe>`` (single-field, row or aggregate context)
|
||||
- ``{rel_field_id.target_field_id}`` → ``_ct_<safe_rel>__<safe_target>``
|
||||
(cross-table reference, KTD-2, U3)
|
||||
|
||||
Returns:
|
||||
Tuple of (substituted_formula, field_mapping, cross_table_mapping)
|
||||
where field_mapping maps safe_name → field_id and
|
||||
cross_table_mapping maps safe_name → (rel_field_id, target_field_id).
|
||||
"""
|
||||
mapping: dict[str, str] = {}
|
||||
field_mapping: dict[str, str] = {}
|
||||
cross_table_mapping: dict[str, tuple[str, str]] = {}
|
||||
|
||||
def _replace(match: re.Match[str]) -> str:
|
||||
field_id = match.group(1)
|
||||
# Convert UUID-style field_id to a valid Python identifier
|
||||
safe_name = "_f_" + field_id.replace("-", "_")
|
||||
mapping[safe_name] = field_id
|
||||
def _replace_cross(match: re.Match[str]) -> str:
|
||||
rel_field_id = match.group(1)
|
||||
target_field_id = match.group(2)
|
||||
safe_name = "_ct_" + rel_field_id.replace("-", "_") + "__" + target_field_id.replace("-", "_")
|
||||
cross_table_mapping[safe_name] = (rel_field_id, target_field_id)
|
||||
return safe_name
|
||||
|
||||
result = _FIELD_REF_RE.sub(_replace, formula)
|
||||
return result, mapping
|
||||
def _replace_single(match: re.Match[str]) -> str:
|
||||
field_id = match.group(1)
|
||||
safe_name = "_f_" + field_id.replace("-", "_")
|
||||
field_mapping[safe_name] = field_id
|
||||
return safe_name
|
||||
|
||||
# Cross-table refs first (they contain a dot, so won't match _FIELD_REF_RE)
|
||||
result = _CROSS_TABLE_REF_RE.sub(_replace_cross, formula)
|
||||
# Then single-field refs
|
||||
result = _FIELD_REF_RE.sub(_replace_single, result)
|
||||
return result, field_mapping, cross_table_mapping
|
||||
|
||||
|
||||
# ── AST whitelist (KTD7) ──────────────────────────────────
|
||||
|
|
@ -84,6 +119,8 @@ _ALLOWED_NODES: frozenset[type[ast.AST]] = frozenset(
|
|||
ast.Constant,
|
||||
ast.IfExp,
|
||||
ast.Load,
|
||||
ast.List,
|
||||
ast.Tuple,
|
||||
ast.Add,
|
||||
ast.Sub,
|
||||
ast.Mult,
|
||||
|
|
@ -105,6 +142,22 @@ _ALLOWED_NODES: frozenset[type[ast.AST]] = frozenset(
|
|||
)
|
||||
|
||||
|
||||
def _ast_depth(node: ast.AST) -> int:
|
||||
"""Compute the depth of an AST tree (KTD8).
|
||||
|
||||
Uses iterative BFS to avoid recursion limits on wide trees.
|
||||
Returns the maximum nesting depth.
|
||||
"""
|
||||
if not hasattr(node, "__dict__") and not isinstance(node, list):
|
||||
return 1
|
||||
max_depth = 0
|
||||
for child in ast.iter_child_nodes(node):
|
||||
child_depth = _ast_depth(child)
|
||||
if child_depth > max_depth:
|
||||
max_depth = child_depth
|
||||
return max_depth + 1
|
||||
|
||||
|
||||
class _SecurityVisitor(ast.NodeVisitor):
|
||||
"""Visit AST nodes, reject any not in the whitelist (KTD7)."""
|
||||
|
||||
|
|
@ -138,7 +191,7 @@ class _SecurityVisitor(ast.NodeVisitor):
|
|||
|
||||
def parse_formula(
|
||||
formula: str, allowed_functions: set[str] | None = None
|
||||
) -> tuple[ast.Expression, dict[str, str]]:
|
||||
) -> tuple[ast.Expression, dict[str, str], dict[str, tuple[str, str]]]:
|
||||
"""Parse a formula string into a safe AST.
|
||||
|
||||
Args:
|
||||
|
|
@ -147,8 +200,10 @@ def parse_formula(
|
|||
all functions are allowed (used for syntax-only validation).
|
||||
|
||||
Returns:
|
||||
Tuple of (AST expression, field_ref_mapping) where
|
||||
field_ref_mapping maps safe Python identifiers to original field IDs.
|
||||
Tuple of (AST expression, field_ref_mapping, cross_table_mapping)
|
||||
where field_ref_mapping maps safe Python identifiers to original
|
||||
field IDs, and cross_table_mapping maps safe Python identifiers to
|
||||
(relation_field_id, target_field_id) tuples for cross-table refs.
|
||||
|
||||
Raises:
|
||||
FormulaParseError: Syntax error in formula.
|
||||
|
|
@ -162,8 +217,8 @@ def parse_formula(
|
|||
if not expr:
|
||||
raise FormulaParseError("Empty formula")
|
||||
|
||||
# Substitute field references {field_id} → safe_name
|
||||
substituted, field_mapping = _substitute_field_refs(expr)
|
||||
# Substitute field references {field_id} → safe_name, {rel.target} → _ct_...
|
||||
substituted, field_mapping, cross_table_mapping = _substitute_field_refs(expr)
|
||||
|
||||
try:
|
||||
tree = ast.parse(substituted, mode="eval")
|
||||
|
|
@ -179,7 +234,15 @@ def parse_formula(
|
|||
visitor = _SecurityVisitor(allowed)
|
||||
visitor.visit(tree)
|
||||
|
||||
return tree, field_mapping # type: ignore[return-value]
|
||||
# KTD8: depth limit — reject formulas with AST deeper than MAX_AST_DEPTH
|
||||
depth = _ast_depth(tree)
|
||||
if depth > MAX_AST_DEPTH:
|
||||
raise FormulaDepthExceededError(
|
||||
f"Formula AST depth {depth} exceeds limit {MAX_AST_DEPTH}. "
|
||||
"Simplify the formula (e.g. flatten nested IF/IFS chains)."
|
||||
)
|
||||
|
||||
return tree, field_mapping, cross_table_mapping # type: ignore[return-value]
|
||||
|
||||
|
||||
def evaluate_ast(
|
||||
|
|
@ -275,6 +338,12 @@ def _eval_node(
|
|||
args = [_eval_node(a, fields, functions) for a in node.args]
|
||||
return functions[func_name](*args)
|
||||
|
||||
if isinstance(node, ast.List):
|
||||
return [_eval_node(e, fields, functions) for e in node.elts]
|
||||
|
||||
if isinstance(node, ast.Tuple):
|
||||
return tuple(_eval_node(e, fields, functions) for e in node.elts)
|
||||
|
||||
raise FormulaSecurityError(f"Disallowed node during evaluation: {type(node).__name__}")
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -33,7 +33,9 @@ class FieldType(str, Enum):
|
|||
attachment = "attachment"
|
||||
image = "image"
|
||||
formula = "formula"
|
||||
relation = "relation"
|
||||
lookup = "lookup"
|
||||
rollup = "rollup"
|
||||
|
||||
|
||||
class FieldOwner(str, Enum):
|
||||
|
|
@ -62,6 +64,59 @@ class RecalcStatus(str, Enum):
|
|||
error = "error"
|
||||
|
||||
|
||||
class RelationType(str, Enum):
|
||||
"""Type of relation between two tables (R5).
|
||||
|
||||
- ``one_to_one`` / ``one_to_many``: stored as record JSONB ``["rec_id"]``
|
||||
or ``["rec_id1", "rec_id2"]`` arrays (no junction table).
|
||||
- ``many_to_many``: stored in ``bitable_relation_links`` junction table
|
||||
(KTD-1) for indexed reverse queries.
|
||||
"""
|
||||
|
||||
one_to_one = "one_to_one"
|
||||
one_to_many = "one_to_many"
|
||||
many_to_many = "many_to_many"
|
||||
|
||||
|
||||
class AutomationTriggerType(str, Enum):
|
||||
"""Supported automation trigger types (R13)."""
|
||||
|
||||
record_created = "record_created"
|
||||
record_updated = "record_updated"
|
||||
record_deleted = "record_deleted"
|
||||
|
||||
|
||||
class AutomationActionType(str, Enum):
|
||||
"""Supported automation action types (R14)."""
|
||||
|
||||
create_record = "create_record"
|
||||
update_record = "update_record"
|
||||
send_webhook = "send_webhook"
|
||||
send_email = "send_email"
|
||||
send_message = "send_message"
|
||||
|
||||
|
||||
class AutomationStatus(str, Enum):
|
||||
"""Status of an automation rule execution log (R17)."""
|
||||
|
||||
success = "success"
|
||||
error = "error"
|
||||
retrying = "retrying"
|
||||
|
||||
|
||||
class CrossTableDepType(str, Enum):
|
||||
"""Type of cross-table dependency (KTD-3).
|
||||
|
||||
- ``formula``: a formula field references a relation_field.target_field
|
||||
- ``lookup``: a lookup field pulls from a target table
|
||||
- ``rollup``: a rollup field aggregates a target table column
|
||||
"""
|
||||
|
||||
formula = "formula"
|
||||
lookup = "lookup"
|
||||
rollup = "rollup"
|
||||
|
||||
|
||||
class BitableFile(BaseModel):
|
||||
"""Top-level container grouping related tables (多维表格文件).
|
||||
|
||||
|
|
@ -153,7 +208,9 @@ class Field(BaseModel):
|
|||
``config`` varies by ``field_type``:
|
||||
- select/multiselect: ``{"options": [{"label": "...", "value": "..."}]}``
|
||||
- formula: ``{"formula_expr": "=SUM({field_abc})"}``
|
||||
- lookup: ``{"lookup_target": {"table_id": "...", "field_id": "...", "filter_field_id": "...", "filter_value": "..."}}``
|
||||
- relation: ``{"relation_type": "one_to_one|one_to_many|many_to_many", "target_table_id": "...", "is_bidirectional": bool, "reverse_field_id": "..." (optional)}``
|
||||
- lookup: ``{"relation_field_id": "...", "target_field_id": "..."}``
|
||||
- rollup: ``{"relation_field_id": "...", "target_field_id": "...", "aggregation": "sum|count|avg|min|max|unique"}``
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
|
@ -193,7 +250,11 @@ class View(BaseModel):
|
|||
|
||||
|
||||
class RecalcTask(BaseModel):
|
||||
"""An asynchronous formula recalculation task."""
|
||||
"""An asynchronous formula recalculation task.
|
||||
|
||||
``is_cross_table`` marks tasks triggered by a cross-table dependency —
|
||||
e.g. a record change in table A causes a rollup in table B to recalc.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
|
|
@ -205,3 +266,81 @@ class RecalcTask(BaseModel):
|
|||
error_message: str | None = None
|
||||
queued_at: datetime = PydanticField(default_factory=_utcnow)
|
||||
completed_at: datetime | None = None
|
||||
is_cross_table: bool = False
|
||||
|
||||
|
||||
class RelationLink(BaseModel):
|
||||
"""A row in the ``bitable_relation_links`` junction table (KTD-1, R9).
|
||||
|
||||
Used for many_to_many relations. One-to-one / one-to-many relations
|
||||
store their link IDs directly in the record's JSONB ``values`` — those
|
||||
do not create RelationLink rows.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: str
|
||||
relation_field_id: str
|
||||
source_record_id: str
|
||||
target_record_id: str
|
||||
created_at: datetime = PydanticField(default_factory=_utcnow)
|
||||
|
||||
|
||||
class AutomationRule(BaseModel):
|
||||
"""A saved automation rule (R13-R17).
|
||||
|
||||
``trigger_config`` shape varies by trigger type:
|
||||
- record_created: ``{"type": "record_created"}``
|
||||
- record_updated: ``{"type": "record_updated", "field_ids": ["f1"]}`` (field_ids optional)
|
||||
- record_deleted: ``{"type": "record_deleted"}``
|
||||
|
||||
``action_config`` shape varies by action type (see AutomationActionType).
|
||||
``webhook_token`` is set only for inbound-webhook triggered rules.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: str
|
||||
table_id: str
|
||||
name: str
|
||||
trigger_config: dict[str, object] = PydanticField(default_factory=dict)
|
||||
action_config: dict[str, object] = PydanticField(default_factory=dict)
|
||||
enabled: bool = True
|
||||
webhook_token: str | None = None
|
||||
created_at: datetime = PydanticField(default_factory=_utcnow)
|
||||
updated_at: datetime = PydanticField(default_factory=_utcnow)
|
||||
|
||||
|
||||
class AutomationLog(BaseModel):
|
||||
"""Execution log entry for an automation rule (R17)."""
|
||||
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: str
|
||||
automation_id: str
|
||||
trigger_record_id: str | None = None
|
||||
status: AutomationStatus = AutomationStatus.success
|
||||
error_message: str | None = None
|
||||
attempt: int = 1
|
||||
started_at: datetime = PydanticField(default_factory=_utcnow)
|
||||
completed_at: datetime | None = None
|
||||
|
||||
|
||||
class CrossTableDep(BaseModel):
|
||||
"""A materialized cross-table field dependency edge (KTD-3, R11).
|
||||
|
||||
``source_field_id`` is in ``source_table_id`` and depends on
|
||||
``target_field_id`` in ``target_table_id``. When the target field's
|
||||
value changes in any record, dependents are enqueued for cross-table
|
||||
recalc (U6).
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: str
|
||||
source_table_id: str
|
||||
source_field_id: str
|
||||
target_table_id: str
|
||||
target_field_id: str
|
||||
dep_type: CrossTableDepType = CrossTableDepType.formula
|
||||
created_at: datetime = PydanticField(default_factory=_utcnow)
|
||||
|
|
|
|||
|
|
@ -18,10 +18,11 @@ from __future__ import annotations
|
|||
|
||||
import asyncio
|
||||
import logging
|
||||
from decimal import Decimal
|
||||
|
||||
from agentkit.bitable.db import BitableDB
|
||||
from agentkit.bitable.formula.engine import FormulaEngine
|
||||
from agentkit.bitable.models import FieldType, RecalcStatus, RecalcTask
|
||||
from agentkit.bitable.models import Field, FieldType, Record, RecalcStatus, RecalcTask
|
||||
from agentkit.bitable.repository import BitableRepository
|
||||
from agentkit.bitable.service import BitableService
|
||||
|
||||
|
|
@ -32,6 +33,22 @@ _DEFAULT_REAPER_INTERVAL = 300 # 5 minutes
|
|||
_DEFAULT_STALE_THRESHOLD = 600 # 10 minutes
|
||||
|
||||
|
||||
def _to_float(v: object) -> float | None:
|
||||
"""Normalize JSONB numeric values to float (handles Decimal from asyncpg).
|
||||
|
||||
PostgreSQL JSONB returns numbers as ``Decimal``; ``isinstance(v, (int, float))``
|
||||
misses them. ``bool`` is a subclass of ``int`` — excluded so True/False don't
|
||||
pollute numeric aggregations.
|
||||
"""
|
||||
if isinstance(v, bool):
|
||||
return None
|
||||
if isinstance(v, (int, float)):
|
||||
return float(v)
|
||||
if isinstance(v, Decimal):
|
||||
return float(v)
|
||||
return None
|
||||
|
||||
|
||||
class RecalcWorker:
|
||||
"""Background worker that processes formula recalc tasks.
|
||||
|
||||
|
|
@ -181,29 +198,25 @@ class RecalcWorker:
|
|||
called from the worker loop (atomic claim sets it). When called
|
||||
synchronously via ``service.process_recalc_task``, this method
|
||||
marks it calculating first (idempotent — re-marking is harmless).
|
||||
|
||||
Handles three computed field types (U3/U5/U6):
|
||||
- ``formula``: parsed by FormulaEngine; may reference cross-table
|
||||
fields via ``{rel.target}`` syntax (resolved by _resolve_cross_table_values).
|
||||
- ``lookup``: pulls a single related record's field value via a relation.
|
||||
- ``rollup``: aggregates related records' field values (SUM/AVG/COUNT/MIN/MAX).
|
||||
"""
|
||||
# Idempotent: mark calculating (no-op if already calculating via claim).
|
||||
await self._repo.update_recalc_status(task.id, RecalcStatus.calculating)
|
||||
|
||||
try:
|
||||
field = await self._repo.get_field(task.field_id)
|
||||
if field is None or field.field_type != FieldType.formula:
|
||||
if field is None or field.field_type not in (
|
||||
FieldType.formula,
|
||||
FieldType.lookup,
|
||||
FieldType.rollup,
|
||||
):
|
||||
await self._repo.update_recalc_status(
|
||||
task.id, RecalcStatus.error, "Field not found or not a formula"
|
||||
)
|
||||
return
|
||||
|
||||
formula_expr = field.config.get("formula_expr", "")
|
||||
if not formula_expr:
|
||||
await self._repo.update_recalc_status(
|
||||
task.id, RecalcStatus.error, "No formula_expr in field config"
|
||||
)
|
||||
return
|
||||
|
||||
engine = await self._get_or_build_engine(task.table_id)
|
||||
if engine is None:
|
||||
await self._repo.update_recalc_status(
|
||||
task.id, RecalcStatus.error, "No formula fields in table"
|
||||
task.id, RecalcStatus.error, "Field not found or not computed"
|
||||
)
|
||||
return
|
||||
|
||||
|
|
@ -214,18 +227,13 @@ class RecalcWorker:
|
|||
)
|
||||
return
|
||||
|
||||
deps = engine.get_dependencies(task.field_id)
|
||||
column_values: dict[str, list[object]] = {}
|
||||
for dep_field_id in deps:
|
||||
column_values[dep_field_id] = await self._repo.get_column_values(
|
||||
task.table_id, dep_field_id
|
||||
)
|
||||
|
||||
result = engine.evaluate(
|
||||
task.field_id,
|
||||
row_values=record.values,
|
||||
column_values=column_values,
|
||||
)
|
||||
if field.field_type == FieldType.lookup:
|
||||
result = await self._evaluate_lookup(field, record)
|
||||
elif field.field_type == FieldType.rollup:
|
||||
result = await self._evaluate_rollup(field, record)
|
||||
else:
|
||||
# Formula — may have cross-table refs (U3)
|
||||
result = await self._evaluate_formula(field, record, task.table_id)
|
||||
|
||||
await self._repo.set_formula_value(task.record_id, task.field_id, result)
|
||||
await self._repo.update_recalc_status(task.id, RecalcStatus.done)
|
||||
|
|
@ -234,6 +242,151 @@ class RecalcWorker:
|
|||
logger.exception("RecalcWorker: error processing task %s", task.id)
|
||||
await self._repo.update_recalc_status(task.id, RecalcStatus.error, str(e)[:500])
|
||||
|
||||
async def _evaluate_formula(self, field: Field, record: Record, table_id: str) -> object:
|
||||
"""Evaluate a formula field (U1+U3). Resolves cross-table refs."""
|
||||
formula_expr = field.config.get("formula_expr", "")
|
||||
if not formula_expr:
|
||||
raise ValueError("No formula_expr in field config")
|
||||
|
||||
engine = await self._get_or_build_engine(table_id)
|
||||
if engine is None:
|
||||
raise ValueError("No formula fields in table")
|
||||
|
||||
deps = engine.get_dependencies(field.id)
|
||||
column_values: dict[str, list[object]] = {}
|
||||
for dep_field_id in deps:
|
||||
column_values[dep_field_id] = await self._repo.get_column_values(table_id, dep_field_id)
|
||||
|
||||
# U3: resolve cross-table references {rel.target}
|
||||
cross_table_values = await self._resolve_cross_table_values(engine, field.id, record)
|
||||
|
||||
return engine.evaluate(
|
||||
field.id,
|
||||
row_values=record.values,
|
||||
column_values=column_values,
|
||||
cross_table_values=cross_table_values,
|
||||
)
|
||||
|
||||
async def _evaluate_lookup(self, field: Field, record: Record) -> object:
|
||||
"""Evaluate a lookup field (U5, R6).
|
||||
|
||||
Config: ``{relation_field_id, target_field_id}``
|
||||
Resolves the related record(s) via the relation field, then reads
|
||||
the target field value from the first related record.
|
||||
"""
|
||||
rel_field_id = str(field.config.get("relation_field_id", ""))
|
||||
target_field_id = str(field.config.get("target_field_id", ""))
|
||||
if not rel_field_id or not target_field_id:
|
||||
raise ValueError("lookup field missing relation_field_id or target_field_id")
|
||||
|
||||
# Same forward/reverse logic as _evaluate_rollup
|
||||
rel_field = await self._repo.get_field(rel_field_id)
|
||||
reverse = rel_field is not None and rel_field.table_id != record.table_id
|
||||
|
||||
related_ids = await self._service.list_related_records(
|
||||
relation_field_id=rel_field_id,
|
||||
record_id=record.id,
|
||||
reverse=reverse,
|
||||
)
|
||||
if not related_ids:
|
||||
return None
|
||||
# Lookup takes the first related record's value (one_to_one / one_to_many first)
|
||||
target_record = await self._repo.get_record(related_ids[0])
|
||||
if target_record is None:
|
||||
return None
|
||||
raw = target_record.values.get(target_field_id)
|
||||
# Normalize Decimal → float for JSON serialization (set_formula_value uses json.dumps)
|
||||
n = _to_float(raw)
|
||||
return n if n is not None else raw
|
||||
|
||||
async def _evaluate_rollup(self, field: Field, record: Record) -> object:
|
||||
"""Evaluate a rollup field (U5, R8).
|
||||
|
||||
Config: ``{relation_field_id, target_field_id, aggregation}``
|
||||
Aggregation is one of: sum, avg, count, min, max, concat, count_distinct.
|
||||
"""
|
||||
rel_field_id = str(field.config.get("relation_field_id", ""))
|
||||
target_field_id = str(field.config.get("target_field_id", ""))
|
||||
aggregation = str(field.config.get("aggregation", "sum")).lower()
|
||||
if not rel_field_id or not target_field_id:
|
||||
raise ValueError("rollup field missing relation_field_id or target_field_id")
|
||||
|
||||
# Determine forward vs reverse lookup: if the record is on the same
|
||||
# table as the relation field, it's a forward lookup (record is the
|
||||
# source). Otherwise the record is on the target table — reverse lookup.
|
||||
rel_field = await self._repo.get_field(rel_field_id)
|
||||
if rel_field is not None and rel_field.table_id != record.table_id:
|
||||
reverse = True
|
||||
else:
|
||||
reverse = False
|
||||
|
||||
related_ids = await self._service.list_related_records(
|
||||
relation_field_id=rel_field_id,
|
||||
record_id=record.id,
|
||||
reverse=reverse,
|
||||
)
|
||||
if not related_ids:
|
||||
return 0 if aggregation in ("sum", "count") else None
|
||||
|
||||
# Fetch target field values from all related records
|
||||
values: list[object] = []
|
||||
for rid in related_ids:
|
||||
rec = await self._repo.get_record(rid)
|
||||
if rec is not None:
|
||||
values.append(rec.values.get(target_field_id))
|
||||
|
||||
# Apply aggregation
|
||||
# ponytail: JSONB numbers come back as Decimal from asyncpg — _to_float
|
||||
# normalizes them (and excludes bool which is an int subclass).
|
||||
numeric_values = [n for v in values if (n := _to_float(v)) is not None]
|
||||
non_none = [v for v in values if v is not None]
|
||||
|
||||
if aggregation == "sum":
|
||||
return sum(numeric_values)
|
||||
if aggregation == "avg":
|
||||
return sum(numeric_values) / len(numeric_values) if numeric_values else 0
|
||||
if aggregation == "count":
|
||||
return len(non_none)
|
||||
if aggregation == "count_distinct":
|
||||
return len(set(non_none))
|
||||
if aggregation == "min":
|
||||
return min(numeric_values) if numeric_values else None
|
||||
if aggregation == "max":
|
||||
return max(numeric_values) if numeric_values else None
|
||||
if aggregation == "concat":
|
||||
return "".join(str(v) for v in non_none)
|
||||
raise ValueError(f"Unknown rollup aggregation: {aggregation}")
|
||||
|
||||
async def _resolve_cross_table_values(
|
||||
self, engine: FormulaEngine, field_id: str, record: Record
|
||||
) -> dict[str, list[object]]:
|
||||
"""Resolve cross-table references for a formula (U3).
|
||||
|
||||
For each ``{rel_field.target_field}`` ref in the formula:
|
||||
1. Find related records via the relation field on this record.
|
||||
2. Read target_field values from those related records.
|
||||
3. Return mapping safe_name → list of values (for aggregate context).
|
||||
|
||||
If the relation field has no links, returns empty lists for each ref.
|
||||
"""
|
||||
cross_table_mapping = engine.get_cross_table_mapping(field_id)
|
||||
if not cross_table_mapping:
|
||||
return {}
|
||||
|
||||
result: dict[str, list[object]] = {}
|
||||
for safe_name, (rel_field_id, target_field_id) in cross_table_mapping.items():
|
||||
related_ids = await self._service.list_related_records(
|
||||
relation_field_id=rel_field_id,
|
||||
record_id=record.id,
|
||||
)
|
||||
values: list[object] = []
|
||||
for rid in related_ids:
|
||||
rec = await self._repo.get_record(rid)
|
||||
if rec is not None:
|
||||
values.append(rec.values.get(target_field_id))
|
||||
result[safe_name] = values
|
||||
return result
|
||||
|
||||
async def _get_or_build_engine(self, table_id: str) -> FormulaEngine | None:
|
||||
"""Get or build a FormulaEngine for a table.
|
||||
|
||||
|
|
|
|||
|
|
@ -11,27 +11,37 @@ import logging
|
|||
import re
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from sqlalchemy import delete, func, insert, select, text, update
|
||||
from sqlalchemy import case, delete, func, insert, select, text, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
|
||||
from agentkit.bitable.db import (
|
||||
AutomationLogModel,
|
||||
AutomationModel,
|
||||
BitableDB,
|
||||
CrossTableDepModel,
|
||||
FieldModel,
|
||||
FileModel,
|
||||
RecordModel,
|
||||
RecalcQueueModel,
|
||||
RelationLinkModel,
|
||||
TableModel,
|
||||
ViewModel,
|
||||
_uuid_str,
|
||||
)
|
||||
from agentkit.bitable.models import (
|
||||
AutomationLog,
|
||||
AutomationRule,
|
||||
AutomationStatus,
|
||||
BitableFile,
|
||||
CrossTableDep,
|
||||
CrossTableDepType,
|
||||
Field,
|
||||
FieldOwner,
|
||||
FieldType,
|
||||
Record,
|
||||
RecalcStatus,
|
||||
RecalcTask,
|
||||
RelationLink,
|
||||
Table,
|
||||
View,
|
||||
ViewType,
|
||||
|
|
@ -226,13 +236,16 @@ class BitableRepository:
|
|||
if not re.match(r"^[a-f0-9-]{36}$", pk_field_id):
|
||||
raise ValueError(f"Invalid pk_field_id format: {pk_field_id}")
|
||||
index_name = f"ix_bitable_records_pk_{table_id.replace('-', '_')}"
|
||||
# asyncpg does not support parameter binding in DDL statements (CREATE INDEX),
|
||||
# so pk_field_id is interpolated directly. It is validated as a strict UUID
|
||||
# (hex + hyphens, 36 chars) above, so SQL injection is not possible here.
|
||||
sql = text(
|
||||
f"CREATE UNIQUE INDEX IF NOT EXISTS {index_name} "
|
||||
f"ON bitable.bitable_records (table_id, (values->>'{pk_field_id}')) "
|
||||
f"WHERE values ? :pk_path"
|
||||
f"WHERE values ? '{pk_field_id}'"
|
||||
)
|
||||
async with self._session_factory() as session:
|
||||
await session.execute(sql, {"pk_path": pk_field_id})
|
||||
await session.execute(sql)
|
||||
await session.commit()
|
||||
|
||||
async def delete_table(self, table_id: str) -> bool:
|
||||
|
|
@ -411,6 +424,27 @@ class BitableRepository:
|
|||
await session.commit()
|
||||
return Record.model_validate(entity) if entity else None
|
||||
|
||||
async def merge_record_values(
|
||||
self, record_id: str, values: dict[str, object]
|
||||
) -> Record | None:
|
||||
"""Shallow-merge values into a record's JSONB (``values || :new``).
|
||||
|
||||
Unlike :meth:`update_record_values` (full replace), this preserves
|
||||
existing keys not present in ``values``. Used by relation link
|
||||
management to set a single field without clobbering the rest.
|
||||
"""
|
||||
async with self._session_factory() as session:
|
||||
stmt = (
|
||||
update(RecordModel)
|
||||
.where(RecordModel.id == record_id)
|
||||
.values(values=RecordModel.values.op("||")(values))
|
||||
.returning(RecordModel)
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
entity = result.scalars().first()
|
||||
await session.commit()
|
||||
return Record.model_validate(entity) if entity else None
|
||||
|
||||
async def delete_record(self, record_id: str) -> bool:
|
||||
"""Delete a record."""
|
||||
async with self._session_factory() as session:
|
||||
|
|
@ -518,9 +552,24 @@ class BitableRepository:
|
|||
# ── Recalc Queue ────────────────────────────────────────
|
||||
|
||||
async def enqueue_recalc(
|
||||
self, table_id: str, record_id: str, field_id: str
|
||||
self,
|
||||
table_id: str,
|
||||
record_id: str,
|
||||
field_id: str,
|
||||
is_cross_table: bool = False,
|
||||
) -> RecalcTask | None:
|
||||
"""Enqueue a recalc task. Returns None if duplicate (ON CONFLICT DO NOTHING)."""
|
||||
"""Enqueue a recalc task.
|
||||
|
||||
If a task for ``(record_id, field_id)`` already exists:
|
||||
- ``pending`` / ``calculating`` → left as-is (avoid duplicate processing).
|
||||
- ``done`` / ``error`` → reset to ``pending`` so a new change can
|
||||
trigger a fresh recalc. Without this, the unique constraint would
|
||||
silently swallow the new request and the field would show stale data.
|
||||
|
||||
``is_cross_table`` marks tasks triggered by a cross-table dependency
|
||||
(U6) — distinguishing them from same-table recalc helps the worker
|
||||
load the correct evaluation context.
|
||||
"""
|
||||
async with self._session_factory() as session:
|
||||
stmt = (
|
||||
pg_insert(RecalcQueueModel)
|
||||
|
|
@ -530,8 +579,22 @@ class BitableRepository:
|
|||
record_id=record_id,
|
||||
field_id=field_id,
|
||||
status=RecalcStatus.pending.value,
|
||||
is_cross_table=is_cross_table,
|
||||
)
|
||||
.on_conflict_do_update(
|
||||
constraint="uq_recalc_record_field",
|
||||
set_={
|
||||
"status": case(
|
||||
(
|
||||
RecalcQueueModel.status.in_(
|
||||
[RecalcStatus.done.value, RecalcStatus.error.value]
|
||||
),
|
||||
RecalcStatus.pending.value,
|
||||
),
|
||||
else_=RecalcQueueModel.status,
|
||||
),
|
||||
},
|
||||
)
|
||||
.on_conflict_do_nothing(constraint="uq_recalc_record_field")
|
||||
.returning(RecalcQueueModel)
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
|
|
@ -951,3 +1014,381 @@ class BitableRepository:
|
|||
)
|
||||
result = await session.execute(stmt)
|
||||
return [Record.model_validate(e) for e in result.scalars().all()]
|
||||
|
||||
# ── Relation Links (U4, KTD-1: many_to_many junction) ───
|
||||
|
||||
async def add_relation_link(
|
||||
self,
|
||||
relation_field_id: str,
|
||||
source_record_id: str,
|
||||
target_record_id: str,
|
||||
) -> RelationLink:
|
||||
"""Create a junction-table link for a many_to_many relation."""
|
||||
async with self._session_factory() as session:
|
||||
stmt = (
|
||||
insert(RelationLinkModel)
|
||||
.values(
|
||||
id=_uuid_str(),
|
||||
relation_field_id=relation_field_id,
|
||||
source_record_id=source_record_id,
|
||||
target_record_id=target_record_id,
|
||||
)
|
||||
.returning(RelationLinkModel)
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
entity = result.scalar_one()
|
||||
await session.commit()
|
||||
return RelationLink.model_validate(entity)
|
||||
|
||||
async def remove_relation_link(
|
||||
self,
|
||||
relation_field_id: str,
|
||||
source_record_id: str,
|
||||
target_record_id: str,
|
||||
) -> bool:
|
||||
"""Remove a single junction-table link."""
|
||||
async with self._session_factory() as session:
|
||||
result = await session.execute(
|
||||
delete(RelationLinkModel).where(
|
||||
RelationLinkModel.relation_field_id == relation_field_id,
|
||||
RelationLinkModel.source_record_id == source_record_id,
|
||||
RelationLinkModel.target_record_id == target_record_id,
|
||||
)
|
||||
)
|
||||
await session.commit()
|
||||
return result.rowcount > 0
|
||||
|
||||
async def remove_all_relation_links(
|
||||
self, relation_field_id: str, source_record_id: str | None = None
|
||||
) -> int:
|
||||
"""Remove all links for a relation field (optionally for one source record).
|
||||
|
||||
Used during cascade cleanup when a relation field is deleted, or when
|
||||
a source record is deleted.
|
||||
"""
|
||||
async with self._session_factory() as session:
|
||||
stmt = delete(RelationLinkModel).where(
|
||||
RelationLinkModel.relation_field_id == relation_field_id
|
||||
)
|
||||
if source_record_id is not None:
|
||||
stmt = stmt.where(
|
||||
RelationLinkModel.source_record_id == source_record_id
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
await session.commit()
|
||||
return result.rowcount
|
||||
|
||||
async def list_relation_links(
|
||||
self, relation_field_id: str, source_record_id: str
|
||||
) -> list[str]:
|
||||
"""List target_record_ids linked to source_record_id via relation_field_id."""
|
||||
async with self._session_factory() as session:
|
||||
stmt = (
|
||||
select(RelationLinkModel.target_record_id)
|
||||
.where(
|
||||
RelationLinkModel.relation_field_id == relation_field_id,
|
||||
RelationLinkModel.source_record_id == source_record_id,
|
||||
)
|
||||
.order_by(RelationLinkModel.created_at)
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
return [r[0] for r in result.fetchall()]
|
||||
|
||||
async def list_reverse_relation_links(
|
||||
self, relation_field_id: str, target_record_id: str
|
||||
) -> list[str]:
|
||||
"""List source_record_ids whose relation_field points at target_record_id.
|
||||
|
||||
Used by the cross-table recalc worker to find which records in the
|
||||
source table depend on a changed target record (U6).
|
||||
"""
|
||||
async with self._session_factory() as session:
|
||||
stmt = (
|
||||
select(RelationLinkModel.source_record_id)
|
||||
.where(
|
||||
RelationLinkModel.relation_field_id == relation_field_id,
|
||||
RelationLinkModel.target_record_id == target_record_id,
|
||||
)
|
||||
.order_by(RelationLinkModel.created_at)
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
return [r[0] for r in result.fetchall()]
|
||||
|
||||
async def list_relation_links_for_field(
|
||||
self, relation_field_id: str
|
||||
) -> list[RelationLink]:
|
||||
"""List all links for a relation field (used during cascade delete)."""
|
||||
async with self._session_factory() as session:
|
||||
stmt = (
|
||||
select(RelationLinkModel)
|
||||
.where(RelationLinkModel.relation_field_id == relation_field_id)
|
||||
.order_by(RelationLinkModel.created_at)
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
return [RelationLink.model_validate(e) for e in result.scalars().all()]
|
||||
|
||||
# ── Cross-table dependency graph (U6, KTD-3) ────────────
|
||||
|
||||
async def add_cross_table_dep(
|
||||
self,
|
||||
source_table_id: str,
|
||||
source_field_id: str,
|
||||
target_table_id: str,
|
||||
target_field_id: str,
|
||||
dep_type: CrossTableDepType = CrossTableDepType.formula,
|
||||
) -> CrossTableDep | None:
|
||||
"""Record a cross-table field dependency. Returns None if duplicate."""
|
||||
async with self._session_factory() as session:
|
||||
stmt = (
|
||||
pg_insert(CrossTableDepModel)
|
||||
.values(
|
||||
id=_uuid_str(),
|
||||
source_table_id=source_table_id,
|
||||
source_field_id=source_field_id,
|
||||
target_table_id=target_table_id,
|
||||
target_field_id=target_field_id,
|
||||
dep_type=dep_type.value,
|
||||
)
|
||||
.on_conflict_do_nothing(
|
||||
constraint="uq_cross_table_dep_source_target"
|
||||
)
|
||||
.returning(CrossTableDepModel)
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
entity = result.scalars().first()
|
||||
await session.commit()
|
||||
return CrossTableDep.model_validate(entity) if entity else None
|
||||
|
||||
async def remove_cross_table_deps_for_source(self, source_field_id: str) -> int:
|
||||
"""Remove all dep edges where source_field_id is the dependent.
|
||||
|
||||
Used when a formula/lookup/rollup field is deleted or its config changes
|
||||
— caller re-adds the current edges afterwards.
|
||||
"""
|
||||
async with self._session_factory() as session:
|
||||
result = await session.execute(
|
||||
delete(CrossTableDepModel).where(
|
||||
CrossTableDepModel.source_field_id == source_field_id
|
||||
)
|
||||
)
|
||||
await session.commit()
|
||||
return result.rowcount
|
||||
|
||||
async def find_cross_table_dependents(
|
||||
self, target_table_id: str, target_field_id: str | None = None
|
||||
) -> list[CrossTableDep]:
|
||||
"""Find all (source_table, source_field) pairs that depend on target.
|
||||
|
||||
Called by the recalc worker after a record write: which fields in
|
||||
OTHER tables will need recalc because this field's value changed?
|
||||
"""
|
||||
async with self._session_factory() as session:
|
||||
stmt = select(CrossTableDepModel).where(
|
||||
CrossTableDepModel.target_table_id == target_table_id
|
||||
)
|
||||
if target_field_id is not None:
|
||||
stmt = stmt.where(CrossTableDepModel.target_field_id == target_field_id)
|
||||
result = await session.execute(stmt)
|
||||
return [CrossTableDep.model_validate(e) for e in result.scalars().all()]
|
||||
|
||||
async def list_cross_table_deps_for_source_table(
|
||||
self, source_table_id: str
|
||||
) -> list[CrossTableDep]:
|
||||
"""List all dep edges originating from fields in source_table_id."""
|
||||
async with self._session_factory() as session:
|
||||
stmt = select(CrossTableDepModel).where(
|
||||
CrossTableDepModel.source_table_id == source_table_id
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
return [CrossTableDep.model_validate(e) for e in result.scalars().all()]
|
||||
|
||||
async def list_cross_table_deps_for_target(
|
||||
self, target_table_id: str, target_field_id: str
|
||||
) -> list[CrossTableDep]:
|
||||
"""List all dep edges pointing at (target_table, target_field) (U6 R11).
|
||||
|
||||
Used by ``trigger_cross_table_recalc`` to find downstream source
|
||||
fields that must be recomputed when a target field changes.
|
||||
"""
|
||||
async with self._session_factory() as session:
|
||||
stmt = select(CrossTableDepModel).where(
|
||||
CrossTableDepModel.target_table_id == target_table_id,
|
||||
CrossTableDepModel.target_field_id == target_field_id,
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
return [CrossTableDep.model_validate(e) for e in result.scalars().all()]
|
||||
|
||||
# ── Automations (U7, R16) ───────────────────────────────
|
||||
|
||||
async def create_automation(
|
||||
self,
|
||||
table_id: str,
|
||||
name: str,
|
||||
trigger_config: dict[str, object],
|
||||
action_config: dict[str, object],
|
||||
enabled: bool = True,
|
||||
webhook_token: str | None = None,
|
||||
) -> AutomationRule:
|
||||
"""Create a new automation rule."""
|
||||
async with self._session_factory() as session:
|
||||
stmt = (
|
||||
insert(AutomationModel)
|
||||
.values(
|
||||
id=_uuid_str(),
|
||||
table_id=table_id,
|
||||
name=name,
|
||||
trigger_config=trigger_config,
|
||||
action_config=action_config,
|
||||
enabled=enabled,
|
||||
webhook_token=webhook_token,
|
||||
)
|
||||
.returning(AutomationModel)
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
entity = result.scalar_one()
|
||||
await session.commit()
|
||||
return AutomationRule.model_validate(entity)
|
||||
|
||||
async def get_automation(self, automation_id: str) -> AutomationRule | None:
|
||||
async with self._session_factory() as session:
|
||||
result = await session.execute(
|
||||
select(AutomationModel).where(AutomationModel.id == automation_id)
|
||||
)
|
||||
entity = result.scalars().first()
|
||||
return AutomationRule.model_validate(entity) if entity else None
|
||||
|
||||
async def get_automation_by_webhook_token(self, token: str) -> AutomationRule | None:
|
||||
"""Look up an automation rule by its inbound webhook token.
|
||||
|
||||
Caller must use ``hmac.compare_digest`` when matching user-supplied
|
||||
tokens; this method does an exact equality lookup against the DB
|
||||
indexed column.
|
||||
"""
|
||||
async with self._session_factory() as session:
|
||||
result = await session.execute(
|
||||
select(AutomationModel).where(AutomationModel.webhook_token == token)
|
||||
)
|
||||
entity = result.scalars().first()
|
||||
return AutomationRule.model_validate(entity) if entity else None
|
||||
|
||||
async def list_automations(
|
||||
self, table_id: str, enabled_only: bool = False
|
||||
) -> list[AutomationRule]:
|
||||
async with self._session_factory() as session:
|
||||
stmt = (
|
||||
select(AutomationModel)
|
||||
.where(AutomationModel.table_id == table_id)
|
||||
.order_by(AutomationModel.created_at)
|
||||
)
|
||||
if enabled_only:
|
||||
stmt = stmt.where(AutomationModel.enabled.is_(True))
|
||||
result = await session.execute(stmt)
|
||||
return [AutomationRule.model_validate(e) for e in result.scalars().all()]
|
||||
|
||||
async def update_automation(self, automation_id: str, **kwargs: object) -> AutomationRule | None:
|
||||
async with self._session_factory() as session:
|
||||
stmt = (
|
||||
update(AutomationModel)
|
||||
.where(AutomationModel.id == automation_id)
|
||||
.values(**kwargs)
|
||||
.returning(AutomationModel)
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
entity = result.scalars().first()
|
||||
await session.commit()
|
||||
return AutomationRule.model_validate(entity) if entity else None
|
||||
|
||||
async def delete_automation(self, automation_id: str) -> bool:
|
||||
async with self._session_factory() as session:
|
||||
# Cascade: delete logs first, then the rule.
|
||||
await session.execute(
|
||||
delete(AutomationLogModel).where(AutomationLogModel.automation_id == automation_id)
|
||||
)
|
||||
result = await session.execute(
|
||||
delete(AutomationModel).where(AutomationModel.id == automation_id)
|
||||
)
|
||||
await session.commit()
|
||||
return result.rowcount > 0
|
||||
|
||||
async def list_automations_for_trigger(
|
||||
self,
|
||||
table_id: str,
|
||||
trigger_type: str,
|
||||
) -> list[AutomationRule]:
|
||||
"""List enabled automations in a table that match a trigger type.
|
||||
|
||||
Uses JSONB containment: ``trigger_config->>'type' = :ttype``. The
|
||||
trigger_type arg is parameterized.
|
||||
"""
|
||||
async with self._session_factory() as session:
|
||||
sql = text(
|
||||
"SELECT * FROM bitable.bitable_automations "
|
||||
"WHERE table_id = :table_id AND enabled = TRUE "
|
||||
"AND trigger_config->>'type' = :ttype "
|
||||
"ORDER BY created_at"
|
||||
)
|
||||
result = await session.execute(
|
||||
sql, {"table_id": table_id, "ttype": trigger_type}
|
||||
)
|
||||
return [AutomationRule.model_validate(AutomationModel(**r._mapping)) for r in result.fetchall()]
|
||||
|
||||
# ── Automation Logs (U7, R17) ───────────────────────────
|
||||
|
||||
async def create_automation_log(
|
||||
self,
|
||||
automation_id: str,
|
||||
trigger_record_id: str | None,
|
||||
status: AutomationStatus = AutomationStatus.success,
|
||||
error_message: str | None = None,
|
||||
attempt: int = 1,
|
||||
) -> AutomationLog:
|
||||
async with self._session_factory() as session:
|
||||
stmt = (
|
||||
insert(AutomationLogModel)
|
||||
.values(
|
||||
id=_uuid_str(),
|
||||
automation_id=automation_id,
|
||||
trigger_record_id=trigger_record_id,
|
||||
status=status.value,
|
||||
error_message=error_message,
|
||||
attempt=attempt,
|
||||
)
|
||||
.returning(AutomationLogModel)
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
entity = result.scalar_one()
|
||||
await session.commit()
|
||||
return AutomationLog.model_validate(entity)
|
||||
|
||||
async def update_automation_log(
|
||||
self,
|
||||
log_id: str,
|
||||
status: AutomationStatus,
|
||||
error_message: str | None = None,
|
||||
) -> None:
|
||||
async with self._session_factory() as session:
|
||||
kwargs: dict[str, object] = {"status": status.value}
|
||||
if error_message is not None:
|
||||
kwargs["error_message"] = error_message
|
||||
if status in (AutomationStatus.success, AutomationStatus.error):
|
||||
kwargs["completed_at"] = func.now()
|
||||
stmt = (
|
||||
update(AutomationLogModel)
|
||||
.where(AutomationLogModel.id == log_id)
|
||||
.values(**kwargs)
|
||||
)
|
||||
await session.execute(stmt)
|
||||
await session.commit()
|
||||
|
||||
async def list_automation_logs(
|
||||
self, automation_id: str, limit: int = 50
|
||||
) -> list[AutomationLog]:
|
||||
async with self._session_factory() as session:
|
||||
stmt = (
|
||||
select(AutomationLogModel)
|
||||
.where(AutomationLogModel.automation_id == automation_id)
|
||||
.order_by(AutomationLogModel.started_at.desc())
|
||||
.limit(limit)
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
return [AutomationLog.model_validate(e) for e in result.scalars().all()]
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ Routes and CLI call this layer — never the repository directly.
|
|||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
from datetime import datetime, timezone
|
||||
|
|
@ -17,12 +18,15 @@ from typing import TYPE_CHECKING, TypeAlias
|
|||
from agentkit.bitable.db import BitableDB
|
||||
from agentkit.bitable.models import (
|
||||
DEFAULT_FIELD_TEMPLATES,
|
||||
AutomationLog,
|
||||
AutomationRule,
|
||||
BitableFile,
|
||||
Field,
|
||||
FieldOwner,
|
||||
FieldType,
|
||||
Record,
|
||||
RecalcTask,
|
||||
RelationType,
|
||||
Table,
|
||||
View,
|
||||
ViewType,
|
||||
|
|
@ -37,6 +41,18 @@ if TYPE_CHECKING:
|
|||
|
||||
def invalidate_engine(self, table_id: str) -> None: ...
|
||||
|
||||
class _AutomationEngine(Protocol):
|
||||
"""Structural type for the automation engine's trigger-dispatch surface."""
|
||||
|
||||
async def fire_trigger(
|
||||
self,
|
||||
table_id: str,
|
||||
trigger_type: object,
|
||||
record_id: str | None = None,
|
||||
changed_field_ids: list[str] | None = None,
|
||||
record_values: dict[str, object] | None = None,
|
||||
) -> int: ...
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
|
@ -75,6 +91,8 @@ class BitableService:
|
|||
self._db = db
|
||||
self._repo = BitableRepository(db)
|
||||
self._recalc_worker: _RecalcWorker | None = None # set via set_recalc_worker
|
||||
self._automation_engine: _AutomationEngine | None = None # set via set_automation_engine
|
||||
self._pending_trigger_tasks: set[asyncio.Task[object]] = set()
|
||||
|
||||
@property
|
||||
def repo(self) -> BitableRepository:
|
||||
|
|
@ -89,11 +107,53 @@ class BitableService:
|
|||
"""
|
||||
self._recalc_worker = worker
|
||||
|
||||
def set_automation_engine(self, engine: _AutomationEngine) -> None:
|
||||
"""Register the AutomationEngine for trigger dispatch (U7).
|
||||
|
||||
Called after both service and engine are constructed. When unset,
|
||||
record writes skip trigger evaluation — automations are silent.
|
||||
"""
|
||||
self._automation_engine = engine
|
||||
|
||||
def _invalidate_engine_cache(self, table_id: str) -> None:
|
||||
"""Invalidate the worker's cached formula engine for a table (P1 #5)."""
|
||||
if self._recalc_worker is not None:
|
||||
self._recalc_worker.invalidate_engine(table_id)
|
||||
|
||||
def _fire_automation_trigger(
|
||||
self,
|
||||
table_id: str,
|
||||
trigger_type: str,
|
||||
record_id: str | None = None,
|
||||
changed_field_ids: list[str] | None = None,
|
||||
record_values: dict[str, object] | None = None,
|
||||
) -> None:
|
||||
"""Fire automation triggers (U7). Best-effort — errors are logged, not raised."""
|
||||
if self._automation_engine is None:
|
||||
return
|
||||
from agentkit.bitable.models import AutomationTriggerType
|
||||
|
||||
try:
|
||||
trig = AutomationTriggerType(trigger_type)
|
||||
except ValueError:
|
||||
return
|
||||
# Schedule the trigger fire — don't block the record write.
|
||||
# Track the task so exceptions surface in the app logger instead of
|
||||
# being silently dropped as "Task exception was never retrieved".
|
||||
import asyncio
|
||||
|
||||
task = asyncio.create_task(
|
||||
self._automation_engine.fire_trigger(
|
||||
table_id=table_id,
|
||||
trigger_type=trig,
|
||||
record_id=record_id,
|
||||
changed_field_ids=changed_field_ids,
|
||||
record_values=record_values,
|
||||
)
|
||||
)
|
||||
self._pending_trigger_tasks.add(task)
|
||||
task.add_done_callback(self._pending_trigger_tasks.discard)
|
||||
|
||||
# ── Files (R1) ─────────────────────────────────────────
|
||||
|
||||
async def create_file(
|
||||
|
|
@ -212,8 +272,8 @@ class BitableService:
|
|||
config=config or {},
|
||||
owner=owner,
|
||||
)
|
||||
# New formula field changes the table's DAG — invalidate cached engine (P1 #5).
|
||||
if field_type == FieldType.formula:
|
||||
# New computed field (formula/lookup/rollup) changes the table's DAG — invalidate cached engine (P1 #5).
|
||||
if field_type in (FieldType.formula, FieldType.lookup, FieldType.rollup):
|
||||
self._invalidate_engine_cache(table_id)
|
||||
return field
|
||||
|
||||
|
|
@ -310,6 +370,12 @@ class BitableService:
|
|||
values[f.id] = now_iso
|
||||
record = await self._repo.create_record(table_id, values)
|
||||
await self._trigger_recalc_for_affected_fields(table_id, record.id)
|
||||
self._fire_automation_trigger(
|
||||
table_id=table_id,
|
||||
trigger_type="record_created",
|
||||
record_id=record.id,
|
||||
record_values=dict(record.values),
|
||||
)
|
||||
return record
|
||||
|
||||
async def create_records_batch(
|
||||
|
|
@ -368,10 +434,22 @@ class BitableService:
|
|||
)
|
||||
|
||||
async def update_record_values(self, record_id: str, values: BitableRecord) -> Record | None:
|
||||
"""Update a record's values (full replace). Triggers recalc for affected formulas."""
|
||||
record = await self._repo.update_record_values(record_id, values)
|
||||
"""Update specific field values in a record (shallow merge, not full replace).
|
||||
|
||||
Triggers recalc for affected formula fields on this table AND on tables
|
||||
that reference this record via relations (cross-table rollup/lookup).
|
||||
"""
|
||||
record = await self._repo.merge_record_values(record_id, values)
|
||||
if record is not None:
|
||||
await self._trigger_recalc_for_affected_fields(record.table_id, record.id)
|
||||
await self._trigger_cross_table_recalc(record)
|
||||
self._fire_automation_trigger(
|
||||
table_id=record.table_id,
|
||||
trigger_type="record_updated",
|
||||
record_id=record.id,
|
||||
changed_field_ids=list(values.keys()),
|
||||
record_values=dict(record.values),
|
||||
)
|
||||
return record
|
||||
|
||||
async def delete_record(self, record_id: str) -> bool:
|
||||
|
|
@ -380,7 +458,15 @@ class BitableService:
|
|||
if record is None:
|
||||
return False
|
||||
await self._cleanup_attachment_files(record.table_id, [record])
|
||||
return await self._repo.delete_record(record_id)
|
||||
deleted = await self._repo.delete_record(record_id)
|
||||
if deleted:
|
||||
self._fire_automation_trigger(
|
||||
table_id=record.table_id,
|
||||
trigger_type="record_deleted",
|
||||
record_id=record_id,
|
||||
record_values=dict(record.values),
|
||||
)
|
||||
return deleted
|
||||
|
||||
async def delete_records_by_table(self, table_id: str) -> int:
|
||||
"""Delete all records in a table and clean up attachment/image files.
|
||||
|
|
@ -576,20 +662,47 @@ class BitableService:
|
|||
# ── Recalc (U3: formula recalc pipeline) ────────────────
|
||||
|
||||
async def _trigger_recalc_for_affected_fields(self, table_id: str, record_id: str) -> None:
|
||||
"""Detect formula fields affected by a record write and enqueue recalc.
|
||||
"""Detect computed fields affected by a record write and enqueue recalc.
|
||||
|
||||
Finds all formula fields in the table, checks which ones depend on
|
||||
the fields that were just written, and enqueues recalc tasks.
|
||||
For simplicity (v1), we enqueue recalc for ALL formula fields in the
|
||||
table for this record — the worker will evaluate them in topological
|
||||
order. The ON CONFLICT DO NOTHING constraint deduplicates.
|
||||
Finds all formula/lookup/rollup fields in the table and enqueues
|
||||
recalc tasks. The worker evaluates them in topological order.
|
||||
ON CONFLICT DO NOTHING deduplicates.
|
||||
"""
|
||||
fields = await self._repo.list_fields(table_id)
|
||||
formula_fields = [f for f in fields if f.field_type == FieldType.formula]
|
||||
computed_fields = [
|
||||
f
|
||||
for f in fields
|
||||
if f.field_type in (FieldType.formula, FieldType.lookup, FieldType.rollup)
|
||||
]
|
||||
|
||||
for f in formula_fields:
|
||||
for f in computed_fields:
|
||||
await self._repo.enqueue_recalc(table_id, record_id, f.id)
|
||||
|
||||
async def _trigger_cross_table_recalc(self, record: Record) -> None:
|
||||
"""Trigger recalc on records that depend on ``record`` via relations.
|
||||
|
||||
When a record's values change, rollup/lookup fields on RELATED records
|
||||
(target table) may need recalc. For each relation field on this record's
|
||||
table, find the target records this record links to and trigger their
|
||||
recalc.
|
||||
"""
|
||||
fields = await self._repo.list_fields(record.table_id)
|
||||
relation_fields = [f for f in fields if f.field_type == FieldType.relation]
|
||||
|
||||
for rf in relation_fields:
|
||||
target_table_id = rf.config.get("target_table_id")
|
||||
if not isinstance(target_table_id, str):
|
||||
continue
|
||||
val = record.values.get(rf.id)
|
||||
target_ids: list[str] = []
|
||||
if isinstance(val, str):
|
||||
target_ids = [val]
|
||||
elif isinstance(val, list):
|
||||
target_ids = [v for v in val if isinstance(v, str)]
|
||||
|
||||
for tid in target_ids:
|
||||
await self._trigger_recalc_for_affected_fields(target_table_id, tid)
|
||||
|
||||
async def trigger_recalc(
|
||||
self, table_id: str, record_id: str, field_id: str
|
||||
) -> RecalcTask | None:
|
||||
|
|
@ -619,3 +732,575 @@ class BitableService:
|
|||
worker = RecalcWorker(self._db, self)
|
||||
await worker.process_task(task)
|
||||
worker.invalidate_engine(task.table_id)
|
||||
|
||||
# ── Relations (U4: R5, R7, R8) ──────────────────────────
|
||||
|
||||
async def create_relation_field(
|
||||
self,
|
||||
table_id: str,
|
||||
name: str,
|
||||
target_table_id: str,
|
||||
relation_type: RelationType = RelationType.one_to_many,
|
||||
is_bidirectional: bool = False,
|
||||
reverse_field_name: str | None = None,
|
||||
) -> Field:
|
||||
"""Create a relation field linking this table to a target table.
|
||||
|
||||
For bidirectional relations, a reverse relation field is also created
|
||||
on the target table. The reverse field's ID is stored in this field's
|
||||
config as ``reverse_field_id``.
|
||||
|
||||
Args:
|
||||
table_id: Source table (where the field is created).
|
||||
name: Field name on the source table.
|
||||
target_table_id: Target table to link to.
|
||||
relation_type: one_to_one, one_to_many, or many_to_many.
|
||||
is_bidirectional: If True, create a reverse field on the target table.
|
||||
reverse_field_name: Name for the reverse field (defaults to "<src_table> ↩").
|
||||
|
||||
Returns:
|
||||
The created relation Field on the source table.
|
||||
"""
|
||||
# Accept both RelationType enum and string for API ergonomics
|
||||
if isinstance(relation_type, str):
|
||||
relation_type = RelationType(relation_type)
|
||||
config: dict[str, object] = {
|
||||
"relation_type": relation_type.value,
|
||||
"target_table_id": target_table_id,
|
||||
"is_bidirectional": is_bidirectional,
|
||||
}
|
||||
|
||||
field = await self._repo.create_field(
|
||||
table_id=table_id,
|
||||
name=name,
|
||||
field_type=FieldType.relation,
|
||||
config=config,
|
||||
owner=FieldOwner.user,
|
||||
)
|
||||
|
||||
if is_bidirectional:
|
||||
# Create reverse field on target table
|
||||
src_table = await self._repo.get_table(table_id)
|
||||
reverse_name = reverse_field_name or f"{src_table.name if src_table else 'Source'} ↩"
|
||||
reverse_config: dict[str, object] = {
|
||||
"relation_type": (
|
||||
RelationType.one_to_many.value
|
||||
if relation_type == RelationType.one_to_many
|
||||
else relation_type.value
|
||||
),
|
||||
"target_table_id": table_id,
|
||||
"is_bidirectional": True,
|
||||
"reverse_field_id": field.id,
|
||||
}
|
||||
reverse_field = await self._repo.create_field(
|
||||
table_id=target_table_id,
|
||||
name=reverse_name,
|
||||
field_type=FieldType.relation,
|
||||
config=reverse_config,
|
||||
owner=FieldOwner.user,
|
||||
)
|
||||
# Link the two fields via reverse_field_id
|
||||
await self._repo.update_field(
|
||||
field.id,
|
||||
config={**config, "reverse_field_id": reverse_field.id},
|
||||
)
|
||||
field = await self._repo.get_field(field.id) or field
|
||||
|
||||
return field
|
||||
|
||||
async def add_relation_link(
|
||||
self,
|
||||
relation_field_id: str,
|
||||
source_record_id: str,
|
||||
target_record_id: str,
|
||||
) -> bool:
|
||||
"""Link two records via a relation field.
|
||||
|
||||
For many_to_many: creates a junction table entry (idempotent).
|
||||
For one_to_one / one_to_many: appends target_record_id to the
|
||||
source record's JSONB values for this field.
|
||||
|
||||
Returns True if a new link was created, False if it already existed.
|
||||
"""
|
||||
field = await self._repo.get_field(relation_field_id)
|
||||
if field is None or field.field_type != FieldType.relation:
|
||||
raise ValueError(f"Field {relation_field_id} is not a relation field")
|
||||
|
||||
relation_type_str = field.config.get("relation_type", "one_to_many")
|
||||
relation_type = RelationType(relation_type_str)
|
||||
|
||||
if relation_type == RelationType.many_to_many:
|
||||
# Junction table entry (idempotent via ON CONFLICT DO NOTHING)
|
||||
added = await self._repo.add_relation_link(
|
||||
relation_field_id=relation_field_id,
|
||||
source_record_id=source_record_id,
|
||||
target_record_id=target_record_id,
|
||||
)
|
||||
if added:
|
||||
# Trigger recalc on both sides
|
||||
src_record = await self._repo.get_record(source_record_id)
|
||||
if src_record is not None:
|
||||
await self._trigger_recalc_for_affected_fields(
|
||||
src_record.table_id, source_record_id
|
||||
)
|
||||
target_table_id = field.config.get("target_table_id")
|
||||
if isinstance(target_table_id, str):
|
||||
await self._trigger_recalc_for_affected_fields(
|
||||
target_table_id, target_record_id
|
||||
)
|
||||
return added
|
||||
|
||||
# one_to_one / one_to_many: store in record JSONB
|
||||
record = await self._repo.get_record(source_record_id)
|
||||
if record is None:
|
||||
raise ValueError(f"Source record {source_record_id} not found")
|
||||
|
||||
current = record.values.get(relation_field_id)
|
||||
if relation_type == RelationType.one_to_one:
|
||||
new_value: object = target_record_id
|
||||
else: # one_to_many
|
||||
existing: list[str] = list(current) if isinstance(current, list) else []
|
||||
if target_record_id in existing:
|
||||
return False
|
||||
existing.append(target_record_id)
|
||||
new_value = existing
|
||||
|
||||
await self._repo.merge_record_values(source_record_id, {relation_field_id: new_value})
|
||||
|
||||
# For bidirectional relations, update the reverse field too
|
||||
reverse_field_id = field.config.get("reverse_field_id")
|
||||
if reverse_field_id and isinstance(reverse_field_id, str):
|
||||
reverse_field = await self._repo.get_field(reverse_field_id)
|
||||
if reverse_field is not None:
|
||||
reverse_record = await self._repo.get_record(target_record_id)
|
||||
if reverse_record is not None:
|
||||
rev_current = reverse_record.values.get(reverse_field_id)
|
||||
rev_existing: list[str] = (
|
||||
list(rev_current) if isinstance(rev_current, list) else []
|
||||
)
|
||||
if source_record_id not in rev_existing:
|
||||
rev_existing.append(source_record_id)
|
||||
await self._repo.merge_record_values(
|
||||
target_record_id,
|
||||
{reverse_field_id: rev_existing},
|
||||
)
|
||||
|
||||
# Trigger recalc on both source and target records — relation links
|
||||
# affect lookup/rollup/formula fields on both sides.
|
||||
await self._trigger_recalc_for_affected_fields(record.table_id, source_record_id)
|
||||
target_table_id = field.config.get("target_table_id")
|
||||
if isinstance(target_table_id, str):
|
||||
await self._trigger_recalc_for_affected_fields(target_table_id, target_record_id)
|
||||
|
||||
return True
|
||||
|
||||
async def remove_relation_link(
|
||||
self,
|
||||
relation_field_id: str,
|
||||
source_record_id: str,
|
||||
target_record_id: str,
|
||||
) -> bool:
|
||||
"""Remove a link between two records.
|
||||
|
||||
Returns True if a link was removed, False if no link existed.
|
||||
For bidirectional relations, the reverse field on the target record is
|
||||
also cleaned up (mirrors ``add_relation_link``).
|
||||
"""
|
||||
field = await self._repo.get_field(relation_field_id)
|
||||
if field is None or field.field_type != FieldType.relation:
|
||||
raise ValueError(f"Field {relation_field_id} is not a relation field")
|
||||
|
||||
relation_type_str = field.config.get("relation_type", "one_to_many")
|
||||
relation_type = RelationType(relation_type_str)
|
||||
|
||||
if relation_type == RelationType.many_to_many:
|
||||
removed = await self._repo.remove_relation_link(
|
||||
relation_field_id=relation_field_id,
|
||||
source_record_id=source_record_id,
|
||||
target_record_id=target_record_id,
|
||||
)
|
||||
if removed:
|
||||
await self._cleanup_reverse_link(field, source_record_id, target_record_id)
|
||||
return removed
|
||||
|
||||
# one_to_one / one_to_many: remove from record JSONB
|
||||
record = await self._repo.get_record(source_record_id)
|
||||
if record is None:
|
||||
return False
|
||||
|
||||
current = record.values.get(relation_field_id)
|
||||
if relation_type == RelationType.one_to_one:
|
||||
if current == target_record_id:
|
||||
await self._repo.merge_record_values(source_record_id, {relation_field_id: None})
|
||||
await self._cleanup_reverse_link(field, source_record_id, target_record_id)
|
||||
return True
|
||||
return False
|
||||
|
||||
# one_to_many: remove from list
|
||||
if not isinstance(current, list):
|
||||
return False
|
||||
if target_record_id not in current:
|
||||
return False
|
||||
new_list = [r for r in current if r != target_record_id]
|
||||
await self._repo.merge_record_values(source_record_id, {relation_field_id: new_list})
|
||||
await self._cleanup_reverse_link(field, source_record_id, target_record_id)
|
||||
return True
|
||||
|
||||
async def _cleanup_reverse_link(
|
||||
self, field: Field, source_record_id: str, target_record_id: str
|
||||
) -> None:
|
||||
"""Remove source_record_id from the target record's reverse field.
|
||||
|
||||
Mirrors the reverse-update block in ``add_relation_link`` so removal
|
||||
stays symmetric for bidirectional relations.
|
||||
"""
|
||||
reverse_field_id = field.config.get("reverse_field_id")
|
||||
if not (reverse_field_id and isinstance(reverse_field_id, str)):
|
||||
return
|
||||
reverse_field = await self._repo.get_field(reverse_field_id)
|
||||
if reverse_field is None:
|
||||
return
|
||||
target_record = await self._repo.get_record(target_record_id)
|
||||
if target_record is None:
|
||||
return
|
||||
rev_current = target_record.values.get(reverse_field_id)
|
||||
if isinstance(rev_current, list):
|
||||
if source_record_id in rev_current:
|
||||
rev_new = [r for r in rev_current if r != source_record_id]
|
||||
await self._repo.merge_record_values(target_record_id, {reverse_field_id: rev_new})
|
||||
elif rev_current == source_record_id:
|
||||
await self._repo.merge_record_values(target_record_id, {reverse_field_id: None})
|
||||
|
||||
async def list_related_records(
|
||||
self,
|
||||
relation_field_id: str,
|
||||
record_id: str,
|
||||
reverse: bool = False,
|
||||
) -> list[str]:
|
||||
"""Get the record IDs related to ``record_id`` via a relation field.
|
||||
|
||||
Args:
|
||||
relation_field_id: The relation field.
|
||||
record_id: The source record ID.
|
||||
reverse: If True, find records that point TO this record
|
||||
(reverse lookup via junction table or reverse field).
|
||||
|
||||
Returns:
|
||||
List of related record IDs.
|
||||
"""
|
||||
field = await self._repo.get_field(relation_field_id)
|
||||
if field is None or field.field_type != FieldType.relation:
|
||||
raise ValueError(f"Field {relation_field_id} is not a relation field")
|
||||
|
||||
relation_type_str = field.config.get("relation_type", "one_to_many")
|
||||
relation_type = RelationType(relation_type_str)
|
||||
|
||||
if relation_type == RelationType.many_to_many:
|
||||
if reverse:
|
||||
links = await self._repo.list_reverse_relation_links(
|
||||
relation_field_id=relation_field_id,
|
||||
target_record_id=record_id,
|
||||
)
|
||||
else:
|
||||
links = await self._repo.list_relation_links(
|
||||
relation_field_id=relation_field_id,
|
||||
source_record_id=record_id,
|
||||
)
|
||||
return [
|
||||
link.target_record_id if not reverse else link.source_record_id for link in links
|
||||
]
|
||||
|
||||
# one_to_one / one_to_many: read from record JSONB
|
||||
if reverse:
|
||||
# Find records in the SOURCE table (where the relation field lives)
|
||||
# that reference this target record via the relation field.
|
||||
source_table_id = field.table_id
|
||||
# ponytail: O(n) scan of source table records. Ceiling: add a
|
||||
# reverse index or use the reverse_field_id for direct lookup.
|
||||
source_records, _ = await self._repo.list_records(source_table_id, limit=10000)
|
||||
result: list[str] = []
|
||||
for rec in source_records:
|
||||
val = rec.values.get(relation_field_id)
|
||||
if val == record_id:
|
||||
result.append(rec.id)
|
||||
elif isinstance(val, list) and record_id in val:
|
||||
result.append(rec.id)
|
||||
return result
|
||||
|
||||
record = await self._repo.get_record(record_id)
|
||||
if record is None:
|
||||
return []
|
||||
current = record.values.get(relation_field_id)
|
||||
if current is None:
|
||||
return []
|
||||
if isinstance(current, list):
|
||||
return [str(r) for r in current]
|
||||
return [str(current)]
|
||||
|
||||
async def delete_relation_field(self, field_id: str) -> bool:
|
||||
"""Delete a relation field and cascade-clean junction entries.
|
||||
|
||||
Removes all relation links associated with this field, then deletes
|
||||
the field itself. For bidirectional relations, the reverse field's
|
||||
``reverse_field_id`` config is cleared.
|
||||
"""
|
||||
field = await self._repo.get_field(field_id)
|
||||
if field is None or field.field_type != FieldType.relation:
|
||||
return False
|
||||
|
||||
# Clean up junction table entries (many_to_many only)
|
||||
await self._repo.remove_all_relation_links(field_id)
|
||||
|
||||
# Clean up record JSONB values (all relation types)
|
||||
await self._repo.remove_field_from_records(field.table_id, field_id)
|
||||
|
||||
# Clear reverse field's reference
|
||||
reverse_field_id = field.config.get("reverse_field_id")
|
||||
if reverse_field_id and isinstance(reverse_field_id, str):
|
||||
reverse_field = await self._repo.get_field(reverse_field_id)
|
||||
if reverse_field is not None:
|
||||
updated_config = {
|
||||
k: v for k, v in reverse_field.config.items() if k != "reverse_field_id"
|
||||
}
|
||||
updated_config["is_bidirectional"] = False
|
||||
await self._repo.update_field(reverse_field_id, config=updated_config)
|
||||
|
||||
return await self._repo.delete_field(field_id)
|
||||
|
||||
# ── Lookup & Rollup (U5: R6, R8) ───────────────────────
|
||||
|
||||
async def create_lookup_field(
|
||||
self,
|
||||
table_id: str,
|
||||
name: str,
|
||||
relation_field_id: str,
|
||||
target_field_id: str,
|
||||
) -> Field:
|
||||
"""Create a lookup field that pulls a value from a related record.
|
||||
|
||||
Config: ``{relation_field_id, target_field_id}``.
|
||||
The recalc worker resolves the relation → reads target_field from
|
||||
the first related record.
|
||||
|
||||
Args:
|
||||
table_id: Table to create the field on.
|
||||
name: Field name.
|
||||
relation_field_id: The relation field on this table to follow.
|
||||
target_field_id: The field on the target table to read.
|
||||
"""
|
||||
config: dict[str, object] = {
|
||||
"relation_field_id": relation_field_id,
|
||||
"target_field_id": target_field_id,
|
||||
}
|
||||
return await self._repo.create_field(
|
||||
table_id=table_id,
|
||||
name=name,
|
||||
field_type=FieldType.lookup,
|
||||
config=config,
|
||||
owner=FieldOwner.user,
|
||||
)
|
||||
|
||||
async def create_rollup_field(
|
||||
self,
|
||||
table_id: str,
|
||||
name: str,
|
||||
relation_field_id: str,
|
||||
target_field_id: str,
|
||||
aggregation: str = "sum",
|
||||
) -> Field:
|
||||
"""Create a rollup field that aggregates related records' values.
|
||||
|
||||
Config: ``{relation_field_id, target_field_id, aggregation}``.
|
||||
Supported aggregations: sum, avg, count, count_distinct, min, max, concat.
|
||||
|
||||
Args:
|
||||
table_id: Table to create the field on.
|
||||
name: Field name.
|
||||
relation_field_id: The relation field on this table to follow.
|
||||
target_field_id: The field on the target table to aggregate.
|
||||
aggregation: Aggregation function name.
|
||||
"""
|
||||
valid_aggs = {"sum", "avg", "count", "count_distinct", "min", "max", "concat"}
|
||||
if aggregation not in valid_aggs:
|
||||
raise ValueError(f"Invalid aggregation: {aggregation}. Must be one of {valid_aggs}")
|
||||
config: dict[str, object] = {
|
||||
"relation_field_id": relation_field_id,
|
||||
"target_field_id": target_field_id,
|
||||
"aggregation": aggregation,
|
||||
}
|
||||
return await self._repo.create_field(
|
||||
table_id=table_id,
|
||||
name=name,
|
||||
field_type=FieldType.rollup,
|
||||
config=config,
|
||||
owner=FieldOwner.user,
|
||||
)
|
||||
|
||||
# ── Cross-table dependency tracking (U6: R10, R11, R12) ─
|
||||
|
||||
async def register_cross_table_dep(
|
||||
self,
|
||||
source_table_id: str,
|
||||
source_field_id: str,
|
||||
target_table_id: str,
|
||||
target_field_id: str,
|
||||
dep_type: str,
|
||||
) -> None:
|
||||
"""Record a cross-table dependency edge (R10).
|
||||
|
||||
When a target field changes, downstream formulas/lookups/rollups in
|
||||
the source table must be recomputed. This materialized dep table
|
||||
avoids scanning all formulas on every write.
|
||||
"""
|
||||
from agentkit.bitable.models import CrossTableDepType
|
||||
|
||||
try:
|
||||
dep_type_enum = CrossTableDepType(dep_type)
|
||||
except ValueError as e:
|
||||
raise ValueError(f"Invalid dep_type: {dep_type}") from e
|
||||
|
||||
await self._repo.add_cross_table_dep(
|
||||
source_table_id=source_table_id,
|
||||
source_field_id=source_field_id,
|
||||
target_table_id=target_table_id,
|
||||
target_field_id=target_field_id,
|
||||
dep_type=dep_type_enum,
|
||||
)
|
||||
|
||||
async def trigger_cross_table_recalc(
|
||||
self, target_table_id: str, target_field_id: str, target_record_ids: list[str]
|
||||
) -> int:
|
||||
"""Enqueue recalc for downstream fields when a target field changes (R11).
|
||||
|
||||
Finds all source fields that depend on (target_table, target_field),
|
||||
then for each source field, finds records in the source table whose
|
||||
relation field links to any of the changed target records, and
|
||||
enqueues recalc for those source records.
|
||||
|
||||
Returns the number of recalc tasks enqueued.
|
||||
"""
|
||||
deps = await self._repo.list_cross_table_deps_for_target(
|
||||
target_table_id=target_table_id,
|
||||
target_field_id=target_field_id,
|
||||
)
|
||||
if not deps:
|
||||
return 0
|
||||
|
||||
count = 0
|
||||
for dep in deps:
|
||||
# Find source records whose relation field points to changed target records
|
||||
source_records = await self._find_source_records_for_targets(
|
||||
dep.source_field_id, target_record_ids
|
||||
)
|
||||
for record_id in source_records:
|
||||
await self._repo.enqueue_recalc(dep.source_table_id, record_id, dep.source_field_id)
|
||||
count += 1
|
||||
return count
|
||||
|
||||
async def _find_source_records_for_targets(
|
||||
self, relation_field_id: str, target_record_ids: list[str]
|
||||
) -> list[str]:
|
||||
"""Find source records linked to any of the target records via a relation field.
|
||||
|
||||
For many_to_many: queries the junction table.
|
||||
For one_to_one/one_to_many: scans source table records (O(n), ponytail ceiling).
|
||||
"""
|
||||
if not target_record_ids:
|
||||
return []
|
||||
|
||||
field = await self._repo.get_field(relation_field_id)
|
||||
if field is None or field.field_type != FieldType.relation:
|
||||
return []
|
||||
|
||||
relation_type_str = field.config.get("relation_type", "one_to_many")
|
||||
if relation_type_str == "many_to_many":
|
||||
# Junction table: find source_record_ids for any of the target_record_ids
|
||||
source_ids: list[str] = []
|
||||
for tid in target_record_ids:
|
||||
links = await self._repo.list_reverse_relation_links(
|
||||
relation_field_id=relation_field_id,
|
||||
target_record_id=tid,
|
||||
)
|
||||
source_ids.extend(link.source_record_id for link in links)
|
||||
return list(set(source_ids))
|
||||
|
||||
# one_to_one / one_to_many: scan source table records for matching values
|
||||
target_set = set(target_record_ids)
|
||||
src_records, _ = await self._repo.list_records(field.table_id, limit=10000)
|
||||
result: list[str] = []
|
||||
for rec in src_records:
|
||||
val = rec.values.get(relation_field_id)
|
||||
if val is None:
|
||||
continue
|
||||
if isinstance(val, list):
|
||||
if any(str(v) in target_set for v in val):
|
||||
result.append(rec.id)
|
||||
elif str(val) in target_set:
|
||||
result.append(rec.id)
|
||||
return result
|
||||
|
||||
# ── Automation rules (U7: R13-R17) ─────────────────────
|
||||
|
||||
async def create_automation(
|
||||
self,
|
||||
table_id: str,
|
||||
name: str,
|
||||
trigger_config: dict[str, object],
|
||||
action_config: dict[str, object],
|
||||
enabled: bool = True,
|
||||
generate_webhook: bool = False,
|
||||
) -> AutomationRule:
|
||||
"""Create an automation rule.
|
||||
|
||||
Args:
|
||||
table_id: Table to watch for triggers.
|
||||
name: Rule name.
|
||||
trigger_config: Trigger definition (type + optional field_ids).
|
||||
action_config: Action definition (type + params).
|
||||
enabled: Whether the rule is active.
|
||||
generate_webhook: If True, generate a webhook_token for inbound triggering.
|
||||
"""
|
||||
webhook_token = None
|
||||
if generate_webhook:
|
||||
from agentkit.bitable.automation import generate_webhook_token
|
||||
|
||||
webhook_token = generate_webhook_token()
|
||||
return await self._repo.create_automation(
|
||||
table_id=table_id,
|
||||
name=name,
|
||||
trigger_config=trigger_config,
|
||||
action_config=action_config,
|
||||
enabled=enabled,
|
||||
webhook_token=webhook_token,
|
||||
)
|
||||
|
||||
async def get_automation(self, automation_id: str) -> AutomationRule | None:
|
||||
return await self._repo.get_automation(automation_id)
|
||||
|
||||
async def list_automations(
|
||||
self, table_id: str, enabled_only: bool = False
|
||||
) -> list[AutomationRule]:
|
||||
"""List automation rules for a table."""
|
||||
return await self._repo.list_automations(table_id=table_id, enabled_only=enabled_only)
|
||||
|
||||
async def update_automation(
|
||||
self, automation_id: str, **kwargs: object
|
||||
) -> AutomationRule | None:
|
||||
return await self._repo.update_automation(automation_id, **kwargs)
|
||||
|
||||
async def delete_automation(self, automation_id: str) -> bool:
|
||||
return await self._repo.delete_automation(automation_id)
|
||||
|
||||
async def list_automation_logs(
|
||||
self, automation_id: str, limit: int = 50
|
||||
) -> list[AutomationLog]:
|
||||
return await self._repo.list_automation_logs(automation_id, limit=limit)
|
||||
|
||||
async def handle_webhook(self, token: str, payload: dict[str, object]) -> bool:
|
||||
"""Handle an inbound webhook — delegates to the automation engine (R16).
|
||||
|
||||
Returns True if a rule matched the token.
|
||||
"""
|
||||
if self._automation_engine is None:
|
||||
return False
|
||||
return await self._automation_engine.handle_webhook(token, payload)
|
||||
|
|
|
|||
|
|
@ -1,5 +1,7 @@
|
|||
"""LLM Gateway - 统一 LLM 调用入口"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
|
|
|
|||
|
|
@ -823,3 +823,309 @@ async def download_file(
|
|||
if not file_path.exists() or not file_path.is_file():
|
||||
raise HTTPException(status_code=404, detail="File not found")
|
||||
return FileResponse(file_path, filename=safe_filename)
|
||||
|
||||
|
||||
# ── Relation / Lookup / Rollup endpoints (U4/U5, R5-R8) ───
|
||||
|
||||
|
||||
class RelationFieldCreate(BaseModel):
|
||||
"""Request body for creating a relation field."""
|
||||
|
||||
name: str
|
||||
target_table_id: str
|
||||
relation_type: str = "one_to_many" # one_to_one | one_to_many | many_to_many
|
||||
is_bidirectional: bool = False
|
||||
reverse_field_name: str | None = None
|
||||
|
||||
|
||||
class RelationLinkCreate(BaseModel):
|
||||
"""Request body for linking two records via a relation field."""
|
||||
|
||||
source_record_id: str
|
||||
target_record_id: str
|
||||
|
||||
|
||||
class LookupFieldCreate(BaseModel):
|
||||
"""Request body for creating a lookup field."""
|
||||
|
||||
name: str
|
||||
relation_field_id: str
|
||||
target_field_id: str
|
||||
|
||||
|
||||
class RollupFieldCreate(BaseModel):
|
||||
"""Request body for creating a rollup field."""
|
||||
|
||||
name: str
|
||||
relation_field_id: str
|
||||
target_field_id: str
|
||||
aggregation: str = "sum" # sum|avg|count|count_distinct|min|max|concat
|
||||
|
||||
|
||||
@router.post("/tables/{table_id}/relation-fields", status_code=201)
|
||||
async def create_relation_field(
|
||||
table_id: str,
|
||||
body: RelationFieldCreate,
|
||||
user: dict = Depends(require_bitable_auth),
|
||||
service: BitableService = Depends(_get_service),
|
||||
) -> dict:
|
||||
"""Create a relation field linking this table to a target table (R5)."""
|
||||
await _check_table_ownership(service, table_id, user)
|
||||
# Also verify ownership of the target table to prevent IDOR via relation
|
||||
await _check_table_ownership(service, body.target_table_id, user)
|
||||
from agentkit.bitable.models import RelationType
|
||||
|
||||
try:
|
||||
rel_type = RelationType(body.relation_type)
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid relation_type: {e}") from e
|
||||
|
||||
field = await service.create_relation_field(
|
||||
table_id=table_id,
|
||||
name=body.name,
|
||||
target_table_id=body.target_table_id,
|
||||
relation_type=rel_type,
|
||||
is_bidirectional=body.is_bidirectional,
|
||||
reverse_field_name=body.reverse_field_name,
|
||||
)
|
||||
return {"field": field.model_dump(mode="json")}
|
||||
|
||||
|
||||
@router.post("/fields/{field_id}/links", status_code=201)
|
||||
async def add_relation_link(
|
||||
field_id: str,
|
||||
body: RelationLinkCreate,
|
||||
user: dict = Depends(require_bitable_auth),
|
||||
service: BitableService = Depends(_get_service),
|
||||
) -> dict:
|
||||
"""Link two records via a relation field (R7)."""
|
||||
field = await service.get_field(field_id)
|
||||
if field is None:
|
||||
raise HTTPException(status_code=404, detail="Field not found")
|
||||
await _check_table_ownership(service, field.table_id, user)
|
||||
created = await service.add_relation_link(
|
||||
relation_field_id=field_id,
|
||||
source_record_id=body.source_record_id,
|
||||
target_record_id=body.target_record_id,
|
||||
)
|
||||
return {"created": created}
|
||||
|
||||
|
||||
@router.delete("/fields/{field_id}/links")
|
||||
async def remove_relation_link(
|
||||
field_id: str,
|
||||
source_record_id: str,
|
||||
target_record_id: str,
|
||||
user: dict = Depends(require_bitable_auth),
|
||||
service: BitableService = Depends(_get_service),
|
||||
) -> dict:
|
||||
"""Remove a link between two records (R7)."""
|
||||
field = await service.get_field(field_id)
|
||||
if field is None:
|
||||
raise HTTPException(status_code=404, detail="Field not found")
|
||||
await _check_table_ownership(service, field.table_id, user)
|
||||
removed = await service.remove_relation_link(
|
||||
relation_field_id=field_id,
|
||||
source_record_id=source_record_id,
|
||||
target_record_id=target_record_id,
|
||||
)
|
||||
return {"removed": removed}
|
||||
|
||||
|
||||
@router.get("/fields/{field_id}/related-records")
|
||||
async def list_related_records(
|
||||
field_id: str,
|
||||
record_id: str,
|
||||
reverse: bool = False,
|
||||
user: dict = Depends(require_bitable_auth),
|
||||
service: BitableService = Depends(_get_service),
|
||||
) -> dict:
|
||||
"""List records related to ``record_id`` via a relation field (R7)."""
|
||||
field = await service.get_field(field_id)
|
||||
if field is None:
|
||||
raise HTTPException(status_code=404, detail="Field not found")
|
||||
await _check_table_ownership(service, field.table_id, user)
|
||||
ids = await service.list_related_records(
|
||||
relation_field_id=field_id,
|
||||
record_id=record_id,
|
||||
reverse=reverse,
|
||||
)
|
||||
return {"record_ids": ids}
|
||||
|
||||
|
||||
@router.post("/tables/{table_id}/lookup-fields", status_code=201)
|
||||
async def create_lookup_field(
|
||||
table_id: str,
|
||||
body: LookupFieldCreate,
|
||||
user: dict = Depends(require_bitable_auth),
|
||||
service: BitableService = Depends(_get_service),
|
||||
) -> dict:
|
||||
"""Create a lookup field (R6)."""
|
||||
await _check_table_ownership(service, table_id, user)
|
||||
field = await service.create_lookup_field(
|
||||
table_id=table_id,
|
||||
name=body.name,
|
||||
relation_field_id=body.relation_field_id,
|
||||
target_field_id=body.target_field_id,
|
||||
)
|
||||
return {"field": field.model_dump(mode="json")}
|
||||
|
||||
|
||||
@router.post("/tables/{table_id}/rollup-fields", status_code=201)
|
||||
async def create_rollup_field(
|
||||
table_id: str,
|
||||
body: RollupFieldCreate,
|
||||
user: dict = Depends(require_bitable_auth),
|
||||
service: BitableService = Depends(_get_service),
|
||||
) -> dict:
|
||||
"""Create a rollup field (R8)."""
|
||||
await _check_table_ownership(service, table_id, user)
|
||||
try:
|
||||
field = await service.create_rollup_field(
|
||||
table_id=table_id,
|
||||
name=body.name,
|
||||
relation_field_id=body.relation_field_id,
|
||||
target_field_id=body.target_field_id,
|
||||
aggregation=body.aggregation,
|
||||
)
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e)) from e
|
||||
return {"field": field.model_dump(mode="json")}
|
||||
|
||||
|
||||
# ── Automation endpoints (U7, R13-R17) ────────────────────
|
||||
|
||||
|
||||
class AutomationCreate(BaseModel):
|
||||
"""Request body for creating an automation rule."""
|
||||
|
||||
name: str
|
||||
trigger_config: dict[str, Any]
|
||||
action_config: dict[str, Any]
|
||||
enabled: bool = True
|
||||
generate_webhook: bool = False
|
||||
|
||||
|
||||
class AutomationUpdate(BaseModel):
|
||||
"""Request body for updating an automation rule.
|
||||
|
||||
Only these fields may be modified — ``table_id`` and ``webhook_token``
|
||||
are intentionally excluded to prevent ownership bypass via field injection.
|
||||
"""
|
||||
|
||||
name: str | None = None
|
||||
trigger_config: dict[str, Any] | None = None
|
||||
action_config: dict[str, Any] | None = None
|
||||
enabled: bool | None = None
|
||||
|
||||
|
||||
@router.post("/tables/{table_id}/automations", status_code=201)
|
||||
async def create_automation(
|
||||
table_id: str,
|
||||
body: AutomationCreate,
|
||||
user: dict = Depends(require_bitable_auth),
|
||||
service: BitableService = Depends(_get_service),
|
||||
) -> dict:
|
||||
"""Create an automation rule (R13-R14)."""
|
||||
await _check_table_ownership(service, table_id, user)
|
||||
rule = await service.create_automation(
|
||||
table_id=table_id,
|
||||
name=body.name,
|
||||
trigger_config=body.trigger_config,
|
||||
action_config=body.action_config,
|
||||
enabled=body.enabled,
|
||||
generate_webhook=body.generate_webhook,
|
||||
)
|
||||
return {"automation": rule.model_dump(mode="json")}
|
||||
|
||||
|
||||
@router.get("/tables/{table_id}/automations")
|
||||
async def list_automations(
|
||||
table_id: str,
|
||||
enabled_only: bool = False,
|
||||
user: dict = Depends(require_bitable_auth),
|
||||
service: BitableService = Depends(_get_service),
|
||||
) -> dict:
|
||||
"""List automation rules for a table."""
|
||||
await _check_table_ownership(service, table_id, user)
|
||||
rules = await service.list_automations(table_id=table_id, enabled_only=enabled_only)
|
||||
return {"automations": [r.model_dump(mode="json") for r in rules]}
|
||||
|
||||
|
||||
@router.get("/automations/{automation_id}")
|
||||
async def get_automation(
|
||||
automation_id: str,
|
||||
user: dict = Depends(require_bitable_auth),
|
||||
service: BitableService = Depends(_get_service),
|
||||
) -> dict:
|
||||
"""Get an automation rule by ID."""
|
||||
rule = await service.get_automation(automation_id)
|
||||
if rule is None:
|
||||
raise HTTPException(status_code=404, detail="Automation not found")
|
||||
await _check_table_ownership(service, rule.table_id, user)
|
||||
return {"automation": rule.model_dump(mode="json")}
|
||||
|
||||
|
||||
@router.patch("/automations/{automation_id}")
|
||||
async def update_automation(
|
||||
automation_id: str,
|
||||
body: AutomationUpdate,
|
||||
user: dict = Depends(require_bitable_auth),
|
||||
service: BitableService = Depends(_get_service),
|
||||
) -> dict:
|
||||
"""Update an automation rule (e.g. enable/disable, change action)."""
|
||||
rule = await service.get_automation(automation_id)
|
||||
if rule is None:
|
||||
raise HTTPException(status_code=404, detail="Automation not found")
|
||||
await _check_table_ownership(service, rule.table_id, user)
|
||||
updates = body.model_dump(exclude_unset=True)
|
||||
updated = await service.update_automation(automation_id, **updates)
|
||||
return {"automation": updated.model_dump(mode="json") if updated else None}
|
||||
|
||||
|
||||
@router.delete("/automations/{automation_id}", status_code=204)
|
||||
async def delete_automation(
|
||||
automation_id: str,
|
||||
user: dict = Depends(require_bitable_auth),
|
||||
service: BitableService = Depends(_get_service),
|
||||
) -> None:
|
||||
"""Delete an automation rule."""
|
||||
rule = await service.get_automation(automation_id)
|
||||
if rule is None:
|
||||
raise HTTPException(status_code=404, detail="Automation not found")
|
||||
await _check_table_ownership(service, rule.table_id, user)
|
||||
await service.delete_automation(automation_id)
|
||||
|
||||
|
||||
@router.get("/automations/{automation_id}/logs")
|
||||
async def list_automation_logs(
|
||||
automation_id: str,
|
||||
limit: int = Query(50, ge=1, le=500),
|
||||
user: dict = Depends(require_bitable_auth),
|
||||
service: BitableService = Depends(_get_service),
|
||||
) -> dict:
|
||||
"""List execution logs for an automation rule (R17)."""
|
||||
rule = await service.get_automation(automation_id)
|
||||
if rule is None:
|
||||
raise HTTPException(status_code=404, detail="Automation not found")
|
||||
await _check_table_ownership(service, rule.table_id, user)
|
||||
logs = await service.list_automation_logs(automation_id, limit=limit)
|
||||
return {"logs": [log.model_dump(mode="json") for log in logs]}
|
||||
|
||||
|
||||
@router.post("/webhooks/{token}")
|
||||
async def inbound_webhook(
|
||||
token: str,
|
||||
body: dict[str, Any],
|
||||
service: BitableService = Depends(_get_service),
|
||||
) -> dict:
|
||||
"""Handle an inbound webhook trigger (R16).
|
||||
|
||||
This endpoint is unauthenticated — the token itself IS the auth.
|
||||
The token is a 256-bit URL-safe random string from ``secrets.token_urlsafe``.
|
||||
"""
|
||||
matched = await service.handle_webhook(token, body)
|
||||
if not matched:
|
||||
# 404 to avoid leaking valid tokens (timing-safe: same response shape)
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
return {"accepted": True}
|
||||
|
|
|
|||
|
|
@ -47,7 +47,10 @@ class BitableTool(Tool):
|
|||
"ingest data from Excel files, databases, or API responses, and "
|
||||
"query records. Actions: create_table, import_excel, "
|
||||
"import_database, collect_api, upsert_records, query_records, "
|
||||
"create_view, update_view, update_field, delete_view."
|
||||
"create_view, update_view, update_field, delete_view, "
|
||||
"create_relation, add_relation_link, remove_relation_link, "
|
||||
"create_lookup_field, create_rollup_field, "
|
||||
"create_automation, validate_formula."
|
||||
),
|
||||
input_schema={
|
||||
"type": "object",
|
||||
|
|
@ -65,6 +68,13 @@ class BitableTool(Tool):
|
|||
"update_view",
|
||||
"update_field",
|
||||
"delete_view",
|
||||
"create_relation",
|
||||
"add_relation_link",
|
||||
"remove_relation_link",
|
||||
"create_lookup_field",
|
||||
"create_rollup_field",
|
||||
"create_automation",
|
||||
"validate_formula",
|
||||
],
|
||||
"description": "Bitable operation to perform.",
|
||||
},
|
||||
|
|
@ -183,6 +193,13 @@ class BitableTool(Tool):
|
|||
"update_view": self._update_view,
|
||||
"update_field": self._update_field,
|
||||
"delete_view": self._delete_view,
|
||||
"create_relation": self._create_relation,
|
||||
"add_relation_link": self._add_relation_link,
|
||||
"remove_relation_link": self._remove_relation_link,
|
||||
"create_lookup_field": self._create_lookup_field,
|
||||
"create_rollup_field": self._create_rollup_field,
|
||||
"create_automation": self._create_automation,
|
||||
"validate_formula": self._validate_formula,
|
||||
}
|
||||
handler = handlers.get(action)
|
||||
if handler is None:
|
||||
|
|
@ -591,3 +608,131 @@ class BitableTool(Tool):
|
|||
resp.raise_for_status()
|
||||
# 204 No Content has an empty body; report a stable success shape.
|
||||
return {"success": True, "deleted": True}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# U4/U5: Relation / Lookup / Rollup actions (R5-R8)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _create_relation(self, **kwargs) -> dict[str, object]:
|
||||
table_id = kwargs.get("table_id")
|
||||
if not table_id:
|
||||
return {"success": False, "error": "Missing required field: table_id"}
|
||||
client = await self._get_client()
|
||||
resp = await client.post(
|
||||
f"/tables/{table_id}/relation-fields",
|
||||
json={
|
||||
"name": kwargs.get("name", ""),
|
||||
"target_table_id": kwargs.get("target_table_id", ""),
|
||||
"relation_type": kwargs.get("relation_type", "one_to_many"),
|
||||
"is_bidirectional": kwargs.get("is_bidirectional", False),
|
||||
"reverse_field_name": kwargs.get("reverse_field_name"),
|
||||
},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return {"success": True, "field": resp.json()["field"]}
|
||||
|
||||
async def _add_relation_link(self, **kwargs) -> dict[str, object]:
|
||||
field_id = kwargs.get("field_id")
|
||||
if not field_id:
|
||||
return {"success": False, "error": "Missing required field: field_id"}
|
||||
client = await self._get_client()
|
||||
resp = await client.post(
|
||||
f"/fields/{field_id}/links",
|
||||
json={
|
||||
"source_record_id": kwargs.get("source_record_id", ""),
|
||||
"target_record_id": kwargs.get("target_record_id", ""),
|
||||
},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return {"success": True, **resp.json()}
|
||||
|
||||
async def _remove_relation_link(self, **kwargs) -> dict[str, object]:
|
||||
field_id = kwargs.get("field_id")
|
||||
if not field_id:
|
||||
return {"success": False, "error": "Missing required field: field_id"}
|
||||
client = await self._get_client()
|
||||
resp = await client.delete(
|
||||
f"/fields/{field_id}/links",
|
||||
params={
|
||||
"source_record_id": kwargs.get("source_record_id", ""),
|
||||
"target_record_id": kwargs.get("target_record_id", ""),
|
||||
},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return {"success": True, **resp.json()}
|
||||
|
||||
async def _create_lookup_field(self, **kwargs) -> dict[str, object]:
|
||||
table_id = kwargs.get("table_id")
|
||||
if not table_id:
|
||||
return {"success": False, "error": "Missing required field: table_id"}
|
||||
client = await self._get_client()
|
||||
resp = await client.post(
|
||||
f"/tables/{table_id}/lookup-fields",
|
||||
json={
|
||||
"name": kwargs.get("name", ""),
|
||||
"relation_field_id": kwargs.get("relation_field_id", ""),
|
||||
"target_field_id": kwargs.get("target_field_id", ""),
|
||||
},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return {"success": True, "field": resp.json()["field"]}
|
||||
|
||||
async def _create_rollup_field(self, **kwargs) -> dict[str, object]:
|
||||
table_id = kwargs.get("table_id")
|
||||
if not table_id:
|
||||
return {"success": False, "error": "Missing required field: table_id"}
|
||||
client = await self._get_client()
|
||||
resp = await client.post(
|
||||
f"/tables/{table_id}/rollup-fields",
|
||||
json={
|
||||
"name": kwargs.get("name", ""),
|
||||
"relation_field_id": kwargs.get("relation_field_id", ""),
|
||||
"target_field_id": kwargs.get("target_field_id", ""),
|
||||
"aggregation": kwargs.get("aggregation", "sum"),
|
||||
},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return {"success": True, "field": resp.json()["field"]}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# U7: Automation actions (R13-R17)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _create_automation(self, **kwargs) -> dict[str, object]:
|
||||
table_id = kwargs.get("table_id")
|
||||
if not table_id:
|
||||
return {"success": False, "error": "Missing required field: table_id"}
|
||||
client = await self._get_client()
|
||||
resp = await client.post(
|
||||
f"/tables/{table_id}/automations",
|
||||
json={
|
||||
"name": kwargs.get("name", ""),
|
||||
"trigger_config": kwargs.get("trigger_config", {}),
|
||||
"action_config": kwargs.get("action_config", {}),
|
||||
"enabled": kwargs.get("enabled", True),
|
||||
"generate_webhook": kwargs.get("generate_webhook", False),
|
||||
},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return {"success": True, "automation": resp.json()["automation"]}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# U3: Formula validation (agent generates formula, validates before save)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _validate_formula(self, **kwargs) -> dict[str, object]:
|
||||
"""Validate a formula expression via the server's parser.
|
||||
|
||||
Agent uses this to check a generated formula before persisting it
|
||||
as a field config — avoids saving broken formulas.
|
||||
"""
|
||||
formula_expr = kwargs.get("formula_expr")
|
||||
if not formula_expr:
|
||||
return {"success": False, "error": "Missing required field: formula_expr"}
|
||||
client = await self._get_client()
|
||||
resp = await client.post(
|
||||
"/fields/validate-formula",
|
||||
json={"formula_expr": formula_expr},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return {"success": True, **resp.json()}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,227 @@
|
|||
"""AE1 acceptance test: cross-table rollup recalculation end-to-end.
|
||||
|
||||
Validates the core v2 scenario from the plan:
|
||||
Orders.amount changes -> Customers.total_amount (rollup SUM) recalculates.
|
||||
|
||||
This test requires PostgreSQL (marked with @pytest.mark.postgres).
|
||||
It exercises the full stack: BitableService -> repository -> recalc_worker
|
||||
-> FormulaEngine -> cross-table relation resolution -> rollup aggregation.
|
||||
|
||||
No mocks — real PG, real async recalc queue, real cross-table deps.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
|
||||
from agentkit.bitable.models import FieldOwner, FieldType
|
||||
|
||||
pytestmark = [pytest.mark.postgres, pytest.mark.asyncio]
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def ae1_setup(bitable_service):
|
||||
"""Set up Customers + Orders tables with a 1:N relation and rollup field.
|
||||
|
||||
Also wires a RecalcWorker into the service so rollup/formula fields
|
||||
actually recalc when records change. Returns a dict with IDs.
|
||||
"""
|
||||
svc = bitable_service
|
||||
|
||||
# Wire a recalc worker (bitable_service fixture doesn't include one)
|
||||
from agentkit.bitable.recalc_worker import RecalcWorker
|
||||
|
||||
worker = RecalcWorker(svc._db, svc)
|
||||
svc.set_recalc_worker(worker)
|
||||
|
||||
# 1. Create Customers table
|
||||
customers_table = await svc.create_table(name="Customers")
|
||||
cust_id_field = await svc._repo.create_field(
|
||||
table_id=customers_table.id,
|
||||
name="cust_id",
|
||||
field_type=FieldType.text,
|
||||
config={},
|
||||
owner=FieldOwner.user,
|
||||
)
|
||||
cust_name_field = await svc._repo.create_field(
|
||||
table_id=customers_table.id,
|
||||
name="name",
|
||||
field_type=FieldType.text,
|
||||
config={},
|
||||
owner=FieldOwner.user,
|
||||
)
|
||||
await svc._repo.update_table(customers_table.id, primary_key_field_id=cust_id_field.id)
|
||||
|
||||
# 2. Create Orders table
|
||||
orders_table = await svc.create_table(name="Orders")
|
||||
ord_id_field = await svc._repo.create_field(
|
||||
table_id=orders_table.id,
|
||||
name="ord_id",
|
||||
field_type=FieldType.text,
|
||||
config={},
|
||||
owner=FieldOwner.user,
|
||||
)
|
||||
ord_amount_field = await svc._repo.create_field(
|
||||
table_id=orders_table.id,
|
||||
name="amount",
|
||||
field_type=FieldType.number,
|
||||
config={},
|
||||
owner=FieldOwner.user,
|
||||
)
|
||||
await svc._repo.update_table(orders_table.id, primary_key_field_id=ord_id_field.id)
|
||||
|
||||
# 3. Create 1:N relation field on Orders -> Customers
|
||||
relation_field = await svc.create_relation_field(
|
||||
table_id=orders_table.id,
|
||||
name="customer",
|
||||
target_table_id=customers_table.id,
|
||||
relation_type="one_to_many",
|
||||
)
|
||||
|
||||
# 4. Create rollup field on Customers: SUM(Orders.amount via relation)
|
||||
rollup_field = await svc.create_rollup_field(
|
||||
table_id=customers_table.id,
|
||||
name="total_amount",
|
||||
relation_field_id=relation_field.id,
|
||||
target_field_id=ord_amount_field.id,
|
||||
aggregation="sum",
|
||||
)
|
||||
|
||||
return {
|
||||
"customers_table": customers_table,
|
||||
"orders_table": orders_table,
|
||||
"cust_id_field": cust_id_field,
|
||||
"cust_name_field": cust_name_field,
|
||||
"ord_id_field": ord_id_field,
|
||||
"ord_amount_field": ord_amount_field,
|
||||
"relation_field": relation_field,
|
||||
"rollup_field": rollup_field,
|
||||
}
|
||||
|
||||
|
||||
async def test_ae1_rollup_sum_recalc_on_insert(bitable_service, ae1_setup) -> None:
|
||||
"""AE1: Inserting Orders updates Customers.total_amount via rollup SUM."""
|
||||
svc = bitable_service
|
||||
cust_fid = ae1_setup["cust_id_field"].id
|
||||
ord_fid = ae1_setup["ord_id_field"].id
|
||||
amt_fid = ae1_setup["ord_amount_field"].id
|
||||
rel_fid = ae1_setup["relation_field"].id
|
||||
rollup_fid = ae1_setup["rollup_field"].id
|
||||
|
||||
# Create a customer
|
||||
cust = await svc.create_record(
|
||||
table_id=ae1_setup["customers_table"].id,
|
||||
values={cust_fid: "c1"},
|
||||
)
|
||||
|
||||
# Create 3 orders linked to the customer
|
||||
for i in range(3):
|
||||
order = await svc.create_record(
|
||||
table_id=ae1_setup["orders_table"].id,
|
||||
values={ord_fid: f"o{i}", amt_fid: (i + 1) * 100},
|
||||
)
|
||||
await svc.add_relation_link(
|
||||
relation_field_id=rel_fid,
|
||||
source_record_id=order.id,
|
||||
target_record_id=cust.id,
|
||||
)
|
||||
|
||||
# Process recalc queue — claim and process all pending tasks
|
||||
worker = svc._recalc_worker
|
||||
if worker is None:
|
||||
pytest.skip("recalc_worker not configured")
|
||||
tasks = await worker._repo.claim_recalc_tasks(limit=50)
|
||||
for task in tasks:
|
||||
await worker.process_task(task)
|
||||
|
||||
# Read the customer record — total_amount should be 100+200+300 = 600
|
||||
updated_cust = await svc._repo.get_record(cust.id)
|
||||
assert updated_cust is not None
|
||||
total = updated_cust.values.get(rollup_fid)
|
||||
assert total == 600, f"Expected total_amount=600, got {total}"
|
||||
|
||||
|
||||
async def test_ae1_rollup_updates_on_order_change(bitable_service, ae1_setup) -> None:
|
||||
"""AE1: Updating an order's amount triggers rollup recalc on customer."""
|
||||
svc = bitable_service
|
||||
cust_fid = ae1_setup["cust_id_field"].id
|
||||
ord_fid = ae1_setup["ord_id_field"].id
|
||||
amt_fid = ae1_setup["ord_amount_field"].id
|
||||
rel_fid = ae1_setup["relation_field"].id
|
||||
rollup_fid = ae1_setup["rollup_field"].id
|
||||
|
||||
cust = await svc.create_record(
|
||||
table_id=ae1_setup["customers_table"].id,
|
||||
values={cust_fid: "c1"},
|
||||
)
|
||||
order = await svc.create_record(
|
||||
table_id=ae1_setup["orders_table"].id,
|
||||
values={ord_fid: "o1", amt_fid: 100},
|
||||
)
|
||||
await svc.add_relation_link(rel_fid, order.id, cust.id)
|
||||
|
||||
# Drain recalc queue
|
||||
worker = svc._recalc_worker
|
||||
if worker is None:
|
||||
pytest.skip("recalc_worker not configured")
|
||||
tasks = await worker._repo.claim_recalc_tasks(limit=50)
|
||||
for task in tasks:
|
||||
await worker.process_task(task)
|
||||
|
||||
# Verify initial sum
|
||||
rec = await svc._repo.get_record(cust.id)
|
||||
assert rec.values.get(rollup_fid) == 100
|
||||
|
||||
# Update the order amount
|
||||
await svc.update_record_values(order.id, {amt_fid: 500})
|
||||
|
||||
# Drain again
|
||||
tasks = await worker._repo.claim_recalc_tasks(limit=50)
|
||||
for task in tasks:
|
||||
await worker.process_task(task)
|
||||
|
||||
# Customer's total should now be 500
|
||||
rec = await svc._repo.get_record(cust.id)
|
||||
assert rec.values.get(rollup_fid) == 500, f"Expected 500 after update, got {rec.values.get(rollup_fid)}"
|
||||
|
||||
|
||||
async def test_ae1_rollup_avg_aggregation(bitable_service, ae1_setup) -> None:
|
||||
"""AE1: Rollup AVG aggregation across related records."""
|
||||
svc = bitable_service
|
||||
cust_fid = ae1_setup["cust_id_field"].id
|
||||
ord_fid = ae1_setup["ord_id_field"].id
|
||||
amt_fid = ae1_setup["ord_amount_field"].id
|
||||
rel_fid = ae1_setup["relation_field"].id
|
||||
|
||||
# Change aggregation to avg — recreate rollup field
|
||||
avg_rollup = await svc.create_rollup_field(
|
||||
table_id=ae1_setup["customers_table"].id,
|
||||
name="avg_amount",
|
||||
relation_field_id=rel_fid,
|
||||
target_field_id=amt_fid,
|
||||
aggregation="avg",
|
||||
)
|
||||
|
||||
cust = await svc.create_record(
|
||||
table_id=ae1_setup["customers_table"].id,
|
||||
values={cust_fid: "c1"},
|
||||
)
|
||||
for amt in [100, 200, 300]:
|
||||
order = await svc.create_record(
|
||||
table_id=ae1_setup["orders_table"].id,
|
||||
values={ord_fid: f"o{amt}", amt_fid: amt},
|
||||
)
|
||||
await svc.add_relation_link(rel_fid, order.id, cust.id)
|
||||
|
||||
# Drain recalc queue
|
||||
worker = svc._recalc_worker
|
||||
if worker is None:
|
||||
pytest.skip("recalc_worker not configured")
|
||||
tasks = await worker._repo.claim_recalc_tasks(limit=50)
|
||||
for task in tasks:
|
||||
await worker.process_task(task)
|
||||
|
||||
rec = await svc._repo.get_record(cust.id)
|
||||
avg = rec.values.get(avg_rollup.id)
|
||||
assert avg == 200, f"Expected avg=200, got {avg}"
|
||||
|
|
@ -0,0 +1,436 @@
|
|||
"""Unit tests for the automation engine (U7, R13-R17).
|
||||
|
||||
Tests the trigger evaluation, action dispatch, retry logic, and webhook
|
||||
handling. These tests do NOT require PostgreSQL — they mock the repo/service
|
||||
interfaces to test the engine logic in isolation.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from agentkit.bitable.automation import AutomationEngine, generate_webhook_token
|
||||
from agentkit.bitable.models import (
|
||||
AutomationRule,
|
||||
AutomationStatus,
|
||||
AutomationTriggerType,
|
||||
)
|
||||
|
||||
|
||||
def _make_rule(
|
||||
rule_id: str = "a1",
|
||||
table_id: str = "t1",
|
||||
trigger_type: str = "record_created",
|
||||
action_type: str = "send_webhook",
|
||||
enabled: bool = True,
|
||||
webhook_token: str | None = None,
|
||||
field_ids: list[str] | None = None,
|
||||
action_config: dict | None = None,
|
||||
) -> AutomationRule:
|
||||
"""Build an AutomationRule for testing."""
|
||||
trig_cfg: dict = {"type": trigger_type}
|
||||
if field_ids:
|
||||
trig_cfg["field_ids"] = field_ids
|
||||
act_cfg = action_config or {"type": action_type, "url": "https://example.com/hook"}
|
||||
return AutomationRule(
|
||||
id=rule_id,
|
||||
table_id=table_id,
|
||||
name=f"Rule {rule_id}",
|
||||
trigger_config=trig_cfg,
|
||||
action_config=act_cfg,
|
||||
enabled=enabled,
|
||||
webhook_token=webhook_token,
|
||||
)
|
||||
|
||||
|
||||
def _make_mock_service() -> tuple[MagicMock, MagicMock]:
|
||||
"""Create a mock service with a mock repo for the engine."""
|
||||
service = MagicMock()
|
||||
repo = MagicMock()
|
||||
service.repo = repo
|
||||
# Async methods on repo
|
||||
repo.list_automations_for_trigger = AsyncMock(return_value=[])
|
||||
repo.get_automation_by_webhook_token = AsyncMock(return_value=None)
|
||||
repo.create_automation_log = AsyncMock(return_value="log1")
|
||||
repo.update_automation_log = AsyncMock(return_value=None)
|
||||
# Async methods on service
|
||||
service.create_record = AsyncMock(return_value=MagicMock(id="r_new"))
|
||||
service.update_record_values = AsyncMock(return_value=None)
|
||||
return service, repo
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Trigger evaluation (R13)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fire_trigger_matches_record_created() -> None:
|
||||
"""record_created trigger fires matching rules."""
|
||||
service, repo = _make_mock_service()
|
||||
rule = _make_rule(trigger_type="record_created")
|
||||
repo.list_automations_for_trigger = AsyncMock(return_value=[rule])
|
||||
|
||||
engine = AutomationEngine(service)
|
||||
matched = await engine.fire_trigger(
|
||||
table_id="t1",
|
||||
trigger_type=AutomationTriggerType.record_created,
|
||||
record_id="r1",
|
||||
record_values={"f1": "value"},
|
||||
)
|
||||
assert matched == 1
|
||||
repo.list_automations_for_trigger.assert_awaited_once_with(
|
||||
table_id="t1", trigger_type="record_created"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fire_trigger_skips_disabled_rules() -> None:
|
||||
"""Disabled rules don't fire."""
|
||||
service, repo = _make_mock_service()
|
||||
rule = _make_rule(enabled=False)
|
||||
repo.list_automations_for_trigger = AsyncMock(return_value=[rule])
|
||||
|
||||
engine = AutomationEngine(service)
|
||||
matched = await engine.fire_trigger(
|
||||
table_id="t1",
|
||||
trigger_type=AutomationTriggerType.record_created,
|
||||
)
|
||||
assert matched == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fire_trigger_record_updated_field_filter() -> None:
|
||||
"""record_updated with field_ids filter only fires on overlap."""
|
||||
service, repo = _make_mock_service()
|
||||
rule_watching_f1 = _make_rule(trigger_type="record_updated", field_ids=["f1"])
|
||||
repo.list_automations_for_trigger = AsyncMock(return_value=[rule_watching_f1])
|
||||
|
||||
engine = AutomationEngine(service)
|
||||
# Changed f2 only — rule watches f1 → no match
|
||||
matched = await engine.fire_trigger(
|
||||
table_id="t1",
|
||||
trigger_type=AutomationTriggerType.record_updated,
|
||||
changed_field_ids=["f2"],
|
||||
)
|
||||
assert matched == 0
|
||||
|
||||
# Changed f1 → match
|
||||
matched = await engine.fire_trigger(
|
||||
table_id="t1",
|
||||
trigger_type=AutomationTriggerType.record_updated,
|
||||
changed_field_ids=["f1", "f2"],
|
||||
)
|
||||
assert matched == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fire_trigger_no_field_filter_matches_any() -> None:
|
||||
"""record_updated without field_ids fires on any update."""
|
||||
service, repo = _make_mock_service()
|
||||
rule_no_filter = _make_rule(trigger_type="record_updated", field_ids=[])
|
||||
repo.list_automations_for_trigger = AsyncMock(return_value=[rule_no_filter])
|
||||
|
||||
engine = AutomationEngine(service)
|
||||
matched = await engine.fire_trigger(
|
||||
table_id="t1",
|
||||
trigger_type=AutomationTriggerType.record_updated,
|
||||
changed_field_ids=["f_anything"],
|
||||
)
|
||||
assert matched == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Webhook inbound (R16)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_handle_webhook_matches_token() -> None:
|
||||
"""Inbound webhook fires the rule with matching token."""
|
||||
service, repo = _make_mock_service()
|
||||
rule = _make_rule(webhook_token="whk_test123")
|
||||
repo.get_automation_by_webhook_token = AsyncMock(return_value=rule)
|
||||
|
||||
engine = AutomationEngine(service)
|
||||
matched = await engine.handle_webhook("whk_test123", {"event": "test"})
|
||||
assert matched is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_handle_webhook_unknown_token() -> None:
|
||||
"""Unknown webhook token returns False (no match)."""
|
||||
service, repo = _make_mock_service()
|
||||
repo.get_automation_by_webhook_token = AsyncMock(return_value=None)
|
||||
|
||||
engine = AutomationEngine(service)
|
||||
matched = await engine.handle_webhook("whk_unknown", {"event": "test"})
|
||||
assert matched is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_handle_webhook_disabled_rule() -> None:
|
||||
"""Disabled rules don't fire via webhook."""
|
||||
service, repo = _make_mock_service()
|
||||
rule = _make_rule(enabled=False, webhook_token="whk_test")
|
||||
repo.get_automation_by_webhook_token = AsyncMock(return_value=rule)
|
||||
|
||||
engine = AutomationEngine(service)
|
||||
matched = await engine.handle_webhook("whk_test", {})
|
||||
assert matched is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Action execution (R14)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_action_create_record_resolves_trigger_placeholders() -> None:
|
||||
"""create_record action resolves @trigger.X placeholders."""
|
||||
service, repo = _make_mock_service()
|
||||
engine = AutomationEngine(service)
|
||||
|
||||
job = {
|
||||
"automation_id": "a1",
|
||||
"table_id": "t1",
|
||||
"trigger_type": "record_created",
|
||||
"record_id": "r_trigger",
|
||||
"record_values": {"f_name": "Alice"},
|
||||
"action_config": {
|
||||
"type": "create_record",
|
||||
"target_table_id": "t2",
|
||||
"values": {
|
||||
"f_target_name": "@trigger.f_name",
|
||||
"f_static": "static_value",
|
||||
},
|
||||
},
|
||||
}
|
||||
await engine._execute_action(job)
|
||||
service.create_record.assert_awaited_once()
|
||||
args, kwargs = service.create_record.call_args
|
||||
assert args[0] == "t2" # target_table_id
|
||||
resolved = args[1]
|
||||
assert resolved["f_target_name"] == "Alice"
|
||||
assert resolved["f_static"] == "static_value"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_action_update_record_defaults_to_trigger_record() -> None:
|
||||
"""update_record defaults target_record_id to the trigger record."""
|
||||
service, repo = _make_mock_service()
|
||||
engine = AutomationEngine(service)
|
||||
|
||||
job = {
|
||||
"automation_id": "a1",
|
||||
"table_id": "t1",
|
||||
"trigger_type": "record_updated",
|
||||
"record_id": "r_trigger",
|
||||
"record_values": {"f_status": "done"},
|
||||
"action_config": {
|
||||
"type": "update_record",
|
||||
"values": {"f_log": "completed"},
|
||||
},
|
||||
}
|
||||
await engine._execute_action(job)
|
||||
service.update_record_values.assert_awaited_once_with("r_trigger", {"f_log": "completed"})
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_action_send_webhook_posts_payload(httpx_mock, monkeypatch) -> None:
|
||||
"""send_webhook POSTs JSON to the configured URL."""
|
||||
# ponytail: uses httpx_mock fixture from pytest-httpx if available;
|
||||
# otherwise this test is skipped.
|
||||
pytest.importorskip("pytest_httpx")
|
||||
# Bypass SSRF host check — this test validates the POST behavior, not the guard.
|
||||
monkeypatch.setattr("agentkit.bitable.automation._assert_safe_host", lambda _host: None)
|
||||
httpx_mock.add_response(url="https://example.com/hook", status_code=200)
|
||||
|
||||
service, repo = _make_mock_service()
|
||||
engine = AutomationEngine(service)
|
||||
|
||||
job = {
|
||||
"automation_id": "a1",
|
||||
"table_id": "t1",
|
||||
"trigger_type": "record_created",
|
||||
"record_id": "r1",
|
||||
"record_values": {"f1": "val"},
|
||||
"action_config": {
|
||||
"type": "send_webhook",
|
||||
"url": "https://example.com/hook",
|
||||
},
|
||||
}
|
||||
await engine._execute_action(job)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_action_unknown_type_raises() -> None:
|
||||
"""Unknown action type raises ValueError."""
|
||||
service, repo = _make_mock_service()
|
||||
engine = AutomationEngine(service)
|
||||
|
||||
job = {
|
||||
"automation_id": "a1",
|
||||
"table_id": "t1",
|
||||
"trigger_type": "record_created",
|
||||
"record_id": "r1",
|
||||
"record_values": {},
|
||||
"action_config": {"type": "invalid_action"},
|
||||
}
|
||||
with pytest.raises(ValueError, match="Unknown action type"):
|
||||
await engine._execute_action(job)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_webhook_ssrf_guard_blocks_loopback() -> None:
|
||||
"""send_webhook rejects loopback URLs (SSRF protection, P1)."""
|
||||
service, repo = _make_mock_service()
|
||||
engine = AutomationEngine(service)
|
||||
|
||||
job = {
|
||||
"automation_id": "a1",
|
||||
"table_id": "t1",
|
||||
"trigger_type": "record_created",
|
||||
"record_id": "r1",
|
||||
"record_values": {"f1": "val"},
|
||||
"action_config": {
|
||||
"type": "send_webhook",
|
||||
"url": "http://127.0.0.1:8080/internal",
|
||||
},
|
||||
}
|
||||
with pytest.raises(ValueError, match="Disallowed|unsafe|private|loopback"):
|
||||
await engine._execute_action(job)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_webhook_ssrf_guard_blocks_private_ip() -> None:
|
||||
"""send_webhook rejects RFC1918 private IPs (SSRF protection, P1)."""
|
||||
service, repo = _make_mock_service()
|
||||
engine = AutomationEngine(service)
|
||||
|
||||
job = {
|
||||
"automation_id": "a2",
|
||||
"table_id": "t1",
|
||||
"trigger_type": "record_created",
|
||||
"record_id": "r1",
|
||||
"record_values": {},
|
||||
"action_config": {
|
||||
"type": "send_webhook",
|
||||
"url": "http://10.0.0.1/admin",
|
||||
},
|
||||
}
|
||||
with pytest.raises(ValueError, match="Disallowed|unsafe|private"):
|
||||
await engine._execute_action(job)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_webhook_ssrf_guard_blocks_non_http_scheme() -> None:
|
||||
"""send_webhook rejects non-HTTP(S) schemes (file://, gopher://, etc.)."""
|
||||
service, repo = _make_mock_service()
|
||||
engine = AutomationEngine(service)
|
||||
|
||||
job = {
|
||||
"automation_id": "a3",
|
||||
"table_id": "t1",
|
||||
"trigger_type": "record_created",
|
||||
"record_id": "r1",
|
||||
"record_values": {},
|
||||
"action_config": {
|
||||
"type": "send_webhook",
|
||||
"url": "file:///etc/passwd",
|
||||
},
|
||||
}
|
||||
with pytest.raises(ValueError, match="scheme"):
|
||||
await engine._execute_action(job)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Retry logic (R15)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retry_succeeds_on_second_attempt(monkeypatch) -> None:
|
||||
"""Action that fails once then succeeds is logged correctly."""
|
||||
service, repo = _make_mock_service()
|
||||
engine = AutomationEngine(service)
|
||||
|
||||
# Make _execute_action fail once, then succeed
|
||||
call_count = [0]
|
||||
|
||||
async def _flaky(job):
|
||||
call_count[0] += 1
|
||||
if call_count[0] == 1:
|
||||
raise RuntimeError("transient failure")
|
||||
|
||||
monkeypatch.setattr(engine, "_execute_action", _flaky)
|
||||
# Speed up retries
|
||||
monkeypatch.setattr("agentkit.bitable.automation._RETRY_DELAYS", [0.0, 0.0, 0.0])
|
||||
|
||||
job = {
|
||||
"automation_id": "a1",
|
||||
"table_id": "t1",
|
||||
"trigger_type": "record_created",
|
||||
"record_id": "r1",
|
||||
"record_values": {},
|
||||
"action_config": {"type": "send_webhook", "url": "x"},
|
||||
}
|
||||
await engine._execute_with_retry(job)
|
||||
assert call_count[0] == 2 # Failed once, succeeded on second
|
||||
# Two log entries created (one per attempt)
|
||||
assert repo.create_automation_log.await_count == 2
|
||||
# Final log status is success
|
||||
repo.update_automation_log.assert_awaited_with(
|
||||
log_id="log1",
|
||||
status=AutomationStatus.success,
|
||||
completed_at=repo.update_automation_log.await_args.kwargs["completed_at"],
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retry_exhausted_after_3_attempts(monkeypatch) -> None:
|
||||
"""Action that always fails is retried 3 times then logged as error."""
|
||||
service, repo = _make_mock_service()
|
||||
engine = AutomationEngine(service)
|
||||
|
||||
async def _always_fail(job):
|
||||
raise RuntimeError("permanent failure")
|
||||
|
||||
monkeypatch.setattr(engine, "_execute_action", _always_fail)
|
||||
monkeypatch.setattr("agentkit.bitable.automation._RETRY_DELAYS", [0.0, 0.0, 0.0])
|
||||
|
||||
job = {
|
||||
"automation_id": "a1",
|
||||
"table_id": "t1",
|
||||
"trigger_type": "record_created",
|
||||
"record_id": "r1",
|
||||
"record_values": {},
|
||||
"action_config": {"type": "send_webhook", "url": "x"},
|
||||
}
|
||||
await engine._execute_with_retry(job)
|
||||
assert repo.create_automation_log.await_count == 3 # 3 attempts
|
||||
# Last log status is error
|
||||
last_call = repo.update_automation_log.await_args_list[-1]
|
||||
assert last_call.kwargs["status"] == AutomationStatus.error
|
||||
assert "permanent failure" in last_call.kwargs["error_message"]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Webhook token generation (R16)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_generate_webhook_token_format() -> None:
|
||||
"""Webhook tokens have the whk_ prefix and sufficient entropy."""
|
||||
token = generate_webhook_token()
|
||||
assert token.startswith("whk_")
|
||||
# token_urlsafe(32) produces ~43 chars; with prefix, ~47
|
||||
assert len(token) > 40
|
||||
|
||||
|
||||
def test_generate_webhook_token_uniqueness() -> None:
|
||||
"""Two generated tokens are different (non-deterministic)."""
|
||||
tokens = {generate_webhook_token() for _ in range(10)}
|
||||
assert len(tokens) == 10
|
||||
|
|
@ -491,17 +491,22 @@ def test_transform_records_missing_keys() -> None:
|
|||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_action_enum_has_10_actions() -> None:
|
||||
"""input_schema.action.enum lists all 10 actions (6 original + 4 new)."""
|
||||
def test_action_enum_has_17_actions() -> None:
|
||||
"""input_schema.action.enum lists all 17 actions (10 original + 7 U4/U5/U7/U3)."""
|
||||
tool = BitableTool(base_url="http://test/api/v1/bitable")
|
||||
actions = tool.input_schema["properties"]["action"]["enum"]
|
||||
assert len(actions) == 10
|
||||
for new_action in ("create_view", "update_view", "update_field", "delete_view"):
|
||||
assert len(actions) == 17
|
||||
for new_action in (
|
||||
"create_view", "update_view", "update_field", "delete_view",
|
||||
"create_relation", "add_relation_link", "remove_relation_link",
|
||||
"create_lookup_field", "create_rollup_field",
|
||||
"create_automation", "validate_formula",
|
||||
):
|
||||
assert new_action in actions
|
||||
|
||||
|
||||
def test_execute_handlers_dict_has_10_actions() -> None:
|
||||
"""execute() handlers dict contains all 10 action keys (KTD10)."""
|
||||
def test_execute_handlers_dict_has_17_actions() -> None:
|
||||
"""execute() handlers dict contains all 17 action keys (KTD10)."""
|
||||
import re
|
||||
|
||||
src = open(
|
||||
|
|
@ -510,8 +515,13 @@ def test_execute_handlers_dict_has_10_actions() -> None:
|
|||
).read()
|
||||
handlers_match = re.search(r"handlers\s*=\s*\{([^}]*)\}", src, re.DOTALL)
|
||||
handler_keys = re.findall(r'"([a-z_]+)":\s*self\._', handlers_match.group(1))
|
||||
assert len(handler_keys) == 10
|
||||
for new_action in ("create_view", "update_view", "update_field", "delete_view"):
|
||||
assert len(handler_keys) == 17
|
||||
for new_action in (
|
||||
"create_view", "update_view", "update_field", "delete_view",
|
||||
"create_relation", "add_relation_link", "remove_relation_link",
|
||||
"create_lookup_field", "create_rollup_field",
|
||||
"create_automation", "validate_formula",
|
||||
):
|
||||
assert new_action in handler_keys
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ import pytest
|
|||
|
||||
|
||||
async def test_init_creates_schema_and_all_tables(bitable_db) -> None:
|
||||
"""init creates the bitable schema and all 7 tables (V2 adds bitable_files)."""
|
||||
"""init creates the bitable schema and all 11 tables (V3 adds 4 new tables)."""
|
||||
from sqlalchemy import text
|
||||
|
||||
async with bitable_db.engine.begin() as conn:
|
||||
|
|
@ -27,7 +27,7 @@ async def test_init_creates_schema_and_all_tables(bitable_db) -> None:
|
|||
)
|
||||
assert result.fetchone() is not None
|
||||
|
||||
# All 7 tables present (V2 adds bitable_files)
|
||||
# All 11 tables present (V2 adds bitable_files; V3 adds 4 new tables)
|
||||
result = await conn.execute(
|
||||
text(
|
||||
"SELECT table_name FROM information_schema.tables "
|
||||
|
|
@ -36,16 +36,85 @@ async def test_init_creates_schema_and_all_tables(bitable_db) -> None:
|
|||
)
|
||||
tables = {row[0] for row in result.fetchall()}
|
||||
assert tables == {
|
||||
"bitable_automation_logs",
|
||||
"bitable_automations",
|
||||
"bitable_cross_table_deps",
|
||||
"bitable_fields",
|
||||
"bitable_files",
|
||||
"bitable_meta",
|
||||
"bitable_records",
|
||||
"bitable_recalc_queue",
|
||||
"bitable_relation_links",
|
||||
"bitable_tables",
|
||||
"bitable_views",
|
||||
}
|
||||
|
||||
|
||||
async def test_v3_migration_adds_is_cross_table_column(bitable_db) -> None:
|
||||
"""V3 migration adds is_cross_table boolean column on recalc_queue."""
|
||||
from sqlalchemy import text
|
||||
|
||||
async with bitable_db.engine.begin() as conn:
|
||||
result = await conn.execute(
|
||||
text(
|
||||
"SELECT column_name, data_type FROM information_schema.columns "
|
||||
"WHERE table_schema = 'bitable' AND table_name = 'bitable_recalc_queue' "
|
||||
"AND column_name = 'is_cross_table'"
|
||||
)
|
||||
)
|
||||
row = result.fetchone()
|
||||
assert row is not None
|
||||
assert row[1] == "boolean"
|
||||
|
||||
|
||||
async def test_v3_migration_creates_relation_links_indexes(bitable_db) -> None:
|
||||
"""V3 creates the two covering indexes on bitable_relation_links."""
|
||||
from sqlalchemy import text
|
||||
|
||||
async with bitable_db.engine.begin() as conn:
|
||||
result = await conn.execute(
|
||||
text(
|
||||
"SELECT indexname FROM pg_indexes "
|
||||
"WHERE schemaname = 'bitable' AND tablename = 'bitable_relation_links'"
|
||||
)
|
||||
)
|
||||
indexes = {row[0] for row in result.fetchall()}
|
||||
assert "ix_relation_links_field_source" in indexes
|
||||
assert "ix_relation_links_field_target" in indexes
|
||||
|
||||
|
||||
async def test_v3_migration_creates_cross_table_deps_unique_constraint(bitable_db) -> None:
|
||||
"""V3 enforces (source_field_id, target_field_id) uniqueness on cross_table_deps."""
|
||||
from sqlalchemy import text
|
||||
|
||||
async with bitable_db.engine.begin() as conn:
|
||||
result = await conn.execute(
|
||||
text(
|
||||
"SELECT conname FROM pg_constraint "
|
||||
"WHERE conrelid = 'bitable.bitable_cross_table_deps'::regclass "
|
||||
"AND conname = 'uq_cross_table_dep_source_target'"
|
||||
)
|
||||
)
|
||||
assert result.fetchone() is not None
|
||||
|
||||
|
||||
async def test_v3_migration_creates_automation_webhook_token_partial_index(bitable_db) -> None:
|
||||
"""V3 creates a partial index on webhook_token (WHERE NOT NULL)."""
|
||||
from sqlalchemy import text
|
||||
|
||||
async with bitable_db.engine.begin() as conn:
|
||||
result = await conn.execute(
|
||||
text(
|
||||
"SELECT indexdef FROM pg_indexes "
|
||||
"WHERE schemaname = 'bitable' AND indexname = 'ix_automations_webhook_token'"
|
||||
)
|
||||
)
|
||||
defn = result.fetchone()
|
||||
assert defn is not None
|
||||
# Partial index must include WHERE webhook_token IS NOT NULL
|
||||
assert "webhook_token IS NOT NULL" in defn[0]
|
||||
|
||||
|
||||
async def test_init_is_idempotent(bitable_db) -> None:
|
||||
"""Calling init() twice does not raise and keeps schema intact."""
|
||||
# bitable_db fixture already called init(); call again
|
||||
|
|
@ -243,3 +312,218 @@ async def test_bitable_db_without_url_raises() -> None:
|
|||
for key, val in zip(("DATABASE_URL", "AGENTKIT_DATABASE_URL"), saved):
|
||||
if val is not None:
|
||||
os.environ[key] = val
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# V3 repository CRUD: relation_links / cross_table_deps / automations / logs
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_relation_links_crud(bitable_db) -> None:
|
||||
"""RelationLink CRUD: add, list forward, list reverse, remove."""
|
||||
from agentkit.bitable.repository import BitableRepository
|
||||
|
||||
repo = BitableRepository(bitable_db)
|
||||
table_a = await repo.create_table(name="Students")
|
||||
table_b = await repo.create_table(name="Courses")
|
||||
# Relation field lives on table_a (Students).
|
||||
rel_field = await repo.create_field(
|
||||
table_id=table_a.id, name="Enrolled Courses", field_type="lookup"
|
||||
)
|
||||
rec_a1 = await repo.create_record(table_id=table_a.id)
|
||||
rec_a2 = await repo.create_record(table_id=table_a.id)
|
||||
rec_b1 = await repo.create_record(table_id=table_b.id)
|
||||
rec_b2 = await repo.create_record(table_id=table_b.id)
|
||||
|
||||
# Add links: a1 → b1, a1 → b2, a2 → b1
|
||||
await repo.add_relation_link(rel_field.id, rec_a1.id, rec_b1.id)
|
||||
await repo.add_relation_link(rel_field.id, rec_a1.id, rec_b2.id)
|
||||
await repo.add_relation_link(rel_field.id, rec_a2.id, rec_b1.id)
|
||||
|
||||
# Forward: a1 → [b1, b2]
|
||||
forward = await repo.list_relation_links(rel_field.id, rec_a1.id)
|
||||
assert set(forward) == {rec_b1.id, rec_b2.id}
|
||||
|
||||
# Reverse: which sources point at b1? → [a1, a2]
|
||||
reverse = await repo.list_reverse_relation_links(rel_field.id, rec_b1.id)
|
||||
assert set(reverse) == {rec_a1.id, rec_a2.id}
|
||||
|
||||
# Remove a1 → b1
|
||||
removed = await repo.remove_relation_link(rel_field.id, rec_a1.id, rec_b1.id)
|
||||
assert removed is True
|
||||
forward_after = await repo.list_relation_links(rel_field.id, rec_a1.id)
|
||||
assert forward_after == [rec_b2.id]
|
||||
|
||||
# remove_all for a source record cascades
|
||||
await repo.add_relation_link(rel_field.id, rec_a1.id, rec_b1.id)
|
||||
count = await repo.remove_all_relation_links(rel_field.id, source_record_id=rec_a1.id)
|
||||
assert count >= 1
|
||||
assert await repo.list_relation_links(rel_field.id, rec_a1.id) == []
|
||||
|
||||
# remove_all for the whole field
|
||||
count = await repo.remove_all_relation_links(rel_field.id)
|
||||
assert count >= 1 # a2→b1 still there
|
||||
|
||||
|
||||
async def test_cross_table_deps_crud(bitable_db) -> None:
|
||||
"""CrossTableDep CRUD: add (dedup), find dependents, remove by source."""
|
||||
from agentkit.bitable.models import CrossTableDepType
|
||||
from agentkit.bitable.repository import BitableRepository
|
||||
|
||||
repo = BitableRepository(bitable_db)
|
||||
table_a = await repo.create_table(name="Orders")
|
||||
table_b = await repo.create_table(name="Customers")
|
||||
source_field = await repo.create_field(
|
||||
table_id=table_b.id, name="total_amount", field_type="rollup"
|
||||
)
|
||||
target_field = await repo.create_field(table_id=table_a.id, name="amount", field_type="number")
|
||||
|
||||
# Add dep edge: source_field (in B) depends on target_field (in A)
|
||||
dep = await repo.add_cross_table_dep(
|
||||
source_table_id=table_b.id,
|
||||
source_field_id=source_field.id,
|
||||
target_table_id=table_a.id,
|
||||
target_field_id=target_field.id,
|
||||
dep_type=CrossTableDepType.rollup,
|
||||
)
|
||||
assert dep is not None
|
||||
|
||||
# Duplicate insert is a no-op
|
||||
dup = await repo.add_cross_table_dep(
|
||||
source_table_id=table_b.id,
|
||||
source_field_id=source_field.id,
|
||||
target_table_id=table_a.id,
|
||||
target_field_id=target_field.id,
|
||||
dep_type=CrossTableDepType.rollup,
|
||||
)
|
||||
assert dup is None
|
||||
|
||||
# Find dependents of table_a / target_field
|
||||
deps = await repo.find_cross_table_dependents(table_a.id, target_field.id)
|
||||
assert len(deps) == 1
|
||||
assert deps[0].source_field_id == source_field.id
|
||||
assert deps[0].dep_type == CrossTableDepType.rollup
|
||||
|
||||
# Remove by source
|
||||
removed = await repo.remove_cross_table_deps_for_source(source_field.id)
|
||||
assert removed == 1
|
||||
assert await repo.find_cross_table_dependents(table_a.id, target_field.id) == []
|
||||
|
||||
|
||||
async def test_automations_crud(bitable_db) -> None:
|
||||
"""AutomationRule CRUD: create, get, list, update, delete (cascade logs)."""
|
||||
from agentkit.bitable.repository import BitableRepository
|
||||
|
||||
repo = BitableRepository(bitable_db)
|
||||
table = await repo.create_table(name="Tasks")
|
||||
|
||||
rule = await repo.create_automation(
|
||||
table_id=table.id,
|
||||
name="Notify on done",
|
||||
trigger_config={"type": "record_updated", "field_ids": ["f_status"]},
|
||||
action_config={"type": "send_webhook", "url": "https://example.com/hook"},
|
||||
)
|
||||
assert rule.enabled is True
|
||||
assert rule.webhook_token is None
|
||||
|
||||
fetched = await repo.get_automation(rule.id)
|
||||
assert fetched is not None and fetched.id == rule.id
|
||||
|
||||
rules = await repo.list_automations(table.id)
|
||||
assert len(rules) == 1
|
||||
|
||||
# list_automations_for_trigger matches on JSONB trigger_config->>'type'
|
||||
matching = await repo.list_automations_for_trigger(table.id, "record_updated")
|
||||
assert len(matching) == 1
|
||||
non_matching = await repo.list_automations_for_trigger(table.id, "record_created")
|
||||
assert non_matching == []
|
||||
|
||||
# update (disable)
|
||||
updated = await repo.update_automation(rule.id, enabled=False)
|
||||
assert updated is not None and updated.enabled is False
|
||||
|
||||
# webhook token lookup
|
||||
rule_with_token = await repo.create_automation(
|
||||
table_id=table.id,
|
||||
name="Inbound webhook",
|
||||
trigger_config={"type": "record_created"},
|
||||
action_config={"type": "create_record"},
|
||||
webhook_token="tok_secret_123",
|
||||
)
|
||||
found = await repo.get_automation_by_webhook_token("tok_secret_123")
|
||||
assert found is not None and found.id == rule_with_token.id
|
||||
|
||||
# delete cascades logs (no logs here, but exercise the path)
|
||||
deleted = await repo.delete_automation(rule.id)
|
||||
assert deleted is True
|
||||
assert await repo.get_automation(rule.id) is None
|
||||
|
||||
|
||||
async def test_automation_logs_crud(bitable_db) -> None:
|
||||
"""AutomationLog CRUD: create, update, list."""
|
||||
from agentkit.bitable.models import AutomationStatus
|
||||
from agentkit.bitable.repository import BitableRepository
|
||||
|
||||
repo = BitableRepository(bitable_db)
|
||||
table = await repo.create_table(name="Tasks")
|
||||
rule = await repo.create_automation(
|
||||
table_id=table.id,
|
||||
name="Auto",
|
||||
trigger_config={"type": "record_created"},
|
||||
action_config={"type": "send_message"},
|
||||
)
|
||||
|
||||
log = await repo.create_automation_log(
|
||||
automation_id=rule.id,
|
||||
trigger_record_id="rec_1",
|
||||
status=AutomationStatus.retrying,
|
||||
attempt=1,
|
||||
)
|
||||
assert log.status == AutomationStatus.retrying
|
||||
|
||||
await repo.update_automation_log(
|
||||
log.id, AutomationStatus.success, error_message=None
|
||||
)
|
||||
|
||||
logs = await repo.list_automation_logs(rule.id)
|
||||
assert len(logs) == 1
|
||||
assert logs[0].status == AutomationStatus.success
|
||||
assert logs[0].completed_at is not None
|
||||
|
||||
|
||||
async def test_enqueue_recalc_with_is_cross_table(bitable_db) -> None:
|
||||
"""enqueue_recalc sets is_cross_table flag when requested (U6)."""
|
||||
from sqlalchemy import text
|
||||
|
||||
from agentkit.bitable.models import FieldType
|
||||
from agentkit.bitable.repository import BitableRepository
|
||||
|
||||
repo = BitableRepository(bitable_db)
|
||||
table = await repo.create_table(name="T")
|
||||
field = await repo.create_field(table_id=table.id, name="f", field_type=FieldType.text)
|
||||
record1 = await repo.create_record(table_id=table.id)
|
||||
record2 = await repo.create_record(table_id=table.id)
|
||||
|
||||
# Same-table recalc — is_cross_table defaults to False
|
||||
task = await repo.enqueue_recalc(table.id, record1.id, field.id)
|
||||
assert task is not None
|
||||
assert task.is_cross_table is False
|
||||
|
||||
# Cross-table recalc — explicit flag set on a different record
|
||||
cross_task = await repo.enqueue_recalc(
|
||||
table.id, record2.id, field.id, is_cross_table=True
|
||||
)
|
||||
assert cross_task is not None
|
||||
assert cross_task.is_cross_table is True
|
||||
|
||||
# Verify the DB column actually stores the flag
|
||||
async with bitable_db.engine.begin() as conn:
|
||||
result = await conn.execute(
|
||||
text(
|
||||
"SELECT is_cross_table FROM bitable.bitable_recalc_queue "
|
||||
"WHERE id = :id"
|
||||
),
|
||||
{"id": cross_task.id},
|
||||
)
|
||||
row = result.fetchone()
|
||||
assert row is not None and row[0] is True
|
||||
|
|
|
|||
|
|
@ -209,3 +209,52 @@ def test_engine_with_uuid_field_ids() -> None:
|
|||
engine.add_formula("calc", f"={{{fid}}} * 2")
|
||||
result = engine.evaluate("calc", row_values={fid: 21})
|
||||
assert result == 42
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# U3: Cross-table formula evaluation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_engine_evaluate_cross_table_aggregate() -> None:
|
||||
"""=SUM({rel.target}) — cross-table aggregate via cross_table_values."""
|
||||
engine = FormulaEngine()
|
||||
engine.add_formula("cross_sum", "=SUM({f_rel.f_target})")
|
||||
# The service layer pre-resolves cross-table refs to value lists
|
||||
# Safe name is _ct_f_rel__f_target
|
||||
cross_values = {"_ct_f_rel__f_target": [10, 20, 30]}
|
||||
result = engine.evaluate(
|
||||
"cross_sum", row_values={}, cross_table_values=cross_values
|
||||
)
|
||||
assert result == 60
|
||||
|
||||
|
||||
def test_engine_cross_table_dep_added_to_dag() -> None:
|
||||
"""Cross-table refs add the local relation field to the DAG."""
|
||||
engine = FormulaEngine()
|
||||
engine.add_formula("calc", "=SUM({f_rel.f_target})")
|
||||
deps = engine.get_dependencies("calc")
|
||||
# The relation field on this table is a dependency (so when the relation
|
||||
# changes, this formula is recomputed)
|
||||
assert "f_rel" in deps
|
||||
|
||||
|
||||
def test_engine_mixed_cross_table_and_row() -> None:
|
||||
"""={f_local} + SUM({f_rel.f_target}) — row + cross-table aggregate."""
|
||||
engine = FormulaEngine()
|
||||
engine.add_formula("mixed", "={f_local} + SUM({f_rel.f_target})")
|
||||
cross_values = {"_ct_f_rel__f_target": [1, 2, 3]}
|
||||
result = engine.evaluate(
|
||||
"mixed", row_values={"f_local": 10}, cross_table_values=cross_values
|
||||
)
|
||||
assert result == 16
|
||||
|
||||
|
||||
def test_engine_cross_table_missing_values_defaults_empty() -> None:
|
||||
"""Missing cross_table_values default to empty list (no crash)."""
|
||||
engine = FormulaEngine()
|
||||
engine.add_formula("calc", "=SUM({f_rel.f_target})")
|
||||
# No cross_table_values provided — should default to empty list
|
||||
result = engine.evaluate("calc", row_values={})
|
||||
# SUM of empty list = 0
|
||||
assert result == 0
|
||||
|
|
|
|||
|
|
@ -9,6 +9,8 @@ from __future__ import annotations
|
|||
import pytest
|
||||
|
||||
from agentkit.bitable.formula.parser import (
|
||||
MAX_AST_DEPTH,
|
||||
FormulaDepthExceededError,
|
||||
FormulaParseError,
|
||||
FormulaSecurityError,
|
||||
UnknownFunctionError,
|
||||
|
|
@ -25,20 +27,20 @@ ALLOWED = {"SUM", "AVG", "COUNT", "MIN", "MAX", "ABS", "ROUND", "IF", "LEN", "CO
|
|||
|
||||
|
||||
def test_parse_simple_arithmetic() -> None:
|
||||
tree, mapping = parse_formula("=1+2*3", ALLOWED)
|
||||
tree, mapping, _ = parse_formula("=1+2*3", ALLOWED)
|
||||
assert mapping == {}
|
||||
result = evaluate_ast(tree, {}, {})
|
||||
assert result == 7
|
||||
|
||||
|
||||
def test_parse_strips_equals_prefix() -> None:
|
||||
tree1, _ = parse_formula("=1+1", ALLOWED)
|
||||
tree2, _ = parse_formula("1+1", ALLOWED)
|
||||
tree1, _, _ = parse_formula("=1+1", ALLOWED)
|
||||
tree2, _, _ = parse_formula("1+1", ALLOWED)
|
||||
assert evaluate_ast(tree1, {}, {}) == evaluate_ast(tree2, {}, {}) == 2
|
||||
|
||||
|
||||
def test_parse_field_reference() -> None:
|
||||
tree, mapping = parse_formula("={field_abc} + 1", ALLOWED)
|
||||
tree, mapping, _ = parse_formula("={field_abc} + 1", ALLOWED)
|
||||
assert "field_abc" in mapping.values()
|
||||
# Safe name is prefixed with _f_
|
||||
safe_name = next(k for k, v in mapping.items() if v == "field_abc")
|
||||
|
|
@ -49,7 +51,7 @@ def test_parse_field_reference() -> None:
|
|||
def test_parse_uuid_field_reference() -> None:
|
||||
"""Field IDs are UUIDs with hyphens — must be substituted to safe names."""
|
||||
fid = "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
|
||||
tree, mapping = parse_formula(f"={{{fid}}} * 2", ALLOWED)
|
||||
tree, mapping, _ = parse_formula(f"={{{fid}}} * 2", ALLOWED)
|
||||
# The mapping should have a safe name → original UUID
|
||||
assert fid in mapping.values()
|
||||
# Evaluate using the safe name (prefixed with _f_)
|
||||
|
|
@ -60,26 +62,26 @@ def test_parse_uuid_field_reference() -> None:
|
|||
|
||||
|
||||
def test_parse_string_concatenation() -> None:
|
||||
tree, _ = parse_formula('="hello" + " " + "world"', ALLOWED)
|
||||
tree, _, _ = parse_formula('="hello" + " " + "world"', ALLOWED)
|
||||
assert evaluate_ast(tree, {}, {}) == "hello world"
|
||||
|
||||
|
||||
def test_parse_conditional_ifexp() -> None:
|
||||
tree, _ = parse_formula("=1 if True else 2", ALLOWED)
|
||||
tree, _, _ = parse_formula("=1 if True else 2", ALLOWED)
|
||||
assert evaluate_ast(tree, {}, {}) == 1
|
||||
|
||||
|
||||
def test_parse_comparison() -> None:
|
||||
tree, mapping = parse_formula("={f} > 5", ALLOWED)
|
||||
tree, mapping, _ = parse_formula("={f} > 5", ALLOWED)
|
||||
safe_name = next(k for k, v in mapping.items() if v == "f")
|
||||
assert evaluate_ast(tree, {safe_name: 10}, {}) is True
|
||||
assert evaluate_ast(tree, {safe_name: 3}, {}) is False
|
||||
|
||||
|
||||
def test_parse_boolean_ops() -> None:
|
||||
tree, _ = parse_formula("=True and False", ALLOWED)
|
||||
tree, _, _ = parse_formula("=True and False", ALLOWED)
|
||||
assert evaluate_ast(tree, {}, {}) is False
|
||||
tree2, _ = parse_formula("=True or False", ALLOWED)
|
||||
tree2, _, _ = parse_formula("=True or False", ALLOWED)
|
||||
assert evaluate_ast(tree2, {}, {}) is True
|
||||
|
||||
|
||||
|
|
@ -89,14 +91,14 @@ def test_parse_boolean_ops() -> None:
|
|||
|
||||
|
||||
def test_parse_function_call_sum() -> None:
|
||||
tree, mapping = parse_formula("=SUM({f1})", ALLOWED)
|
||||
tree, mapping, _ = parse_formula("=SUM({f1})", ALLOWED)
|
||||
safe_name = next(k for k, v in mapping.items() if v == "f1")
|
||||
result = evaluate_ast(tree, {safe_name: [1, 2, 3]}, {"SUM": sum})
|
||||
assert result == 6
|
||||
|
||||
|
||||
def test_parse_function_call_concat() -> None:
|
||||
tree, mapping = parse_formula('=CONCAT({f1}, "-", {f2})', ALLOWED)
|
||||
tree, mapping, _ = parse_formula('=CONCAT({f1}, "-", {f2})', ALLOWED)
|
||||
safe_f1 = next(k for k, v in mapping.items() if v == "f1")
|
||||
safe_f2 = next(k for k, v in mapping.items() if v == "f2")
|
||||
result = evaluate_ast(
|
||||
|
|
@ -106,7 +108,7 @@ def test_parse_function_call_concat() -> None:
|
|||
|
||||
|
||||
def test_parse_nested_function_calls() -> None:
|
||||
tree, _ = parse_formula("=ABS(-5) + ROUND(3.7, 0)", ALLOWED)
|
||||
tree, _, _ = parse_formula("=ABS(-5) + ROUND(3.7, 0)", ALLOWED)
|
||||
funcs = {"ABS": abs, "ROUND": round}
|
||||
result = evaluate_ast(tree, {}, funcs)
|
||||
assert result == 9 # 5 + 4
|
||||
|
|
@ -179,7 +181,7 @@ def test_parse_error_empty_string() -> None:
|
|||
|
||||
|
||||
def test_evaluate_unknown_field_ref_raises() -> None:
|
||||
tree, _ = parse_formula("={nonexistent} + 1", ALLOWED)
|
||||
tree, _, _ = parse_formula("={nonexistent} + 1", ALLOWED)
|
||||
with pytest.raises(FormulaParseError, match="Unknown field reference"):
|
||||
evaluate_ast(tree, {}, {})
|
||||
|
||||
|
|
@ -191,9 +193,117 @@ def test_evaluate_unknown_field_ref_raises() -> None:
|
|||
|
||||
def test_mixed_aggregate_and_row_context() -> None:
|
||||
"""={f1} + SUM({f2}) — row f1 + column f2 sum."""
|
||||
tree, mapping = parse_formula("={f1} + SUM({f2})", ALLOWED)
|
||||
tree, mapping, _ = parse_formula("={f1} + SUM({f2})", ALLOWED)
|
||||
safe_f1 = next(k for k, v in mapping.items() if v == "f1")
|
||||
safe_f2 = next(k for k, v in mapping.items() if v == "f2")
|
||||
# f1 is a row value (scalar), f2 is a column value (list)
|
||||
result = evaluate_ast(tree, {safe_f1: 10, safe_f2: [1, 2, 3]}, {"SUM": sum})
|
||||
assert result == 16 # 10 + 6
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# KTD8: AST depth limit
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_depth_limit_allows_normal_formula() -> None:
|
||||
"""Normal formulas should pass the depth check."""
|
||||
tree, _, _ = parse_formula("=1+2+3+4+5", ALLOWED)
|
||||
assert tree is not None
|
||||
|
||||
|
||||
def test_depth_limit_rejects_deeply_nested() -> None:
|
||||
"""Deeply nested expressions should be rejected (KTD8)."""
|
||||
# Build a formula deeper than MAX_AST_DEPTH: nested additions
|
||||
# Each "x + " adds 1 level of BinOp depth
|
||||
deep_formula = "=" + " + ".join(["1"] * (MAX_AST_DEPTH + 5))
|
||||
with pytest.raises(FormulaDepthExceededError, match="depth"):
|
||||
parse_formula(deep_formula, ALLOWED)
|
||||
|
||||
|
||||
def test_depth_limit_allows_at_boundary() -> None:
|
||||
"""Formula at exactly the limit should pass."""
|
||||
# Build a formula with depth close to but not exceeding MAX_AST_DEPTH
|
||||
# A simple sum of N numbers has depth ~N, but ast groups them differently.
|
||||
# A safer test: shallow formula always passes.
|
||||
tree, _, _ = parse_formula("=SUM({f1}) + IF({f2}, {f3}, {f4})", ALLOWED)
|
||||
assert tree is not None
|
||||
|
||||
|
||||
def test_list_literal_evaluated() -> None:
|
||||
"""List literals [1, 2, 3] should be evaluated as Python lists."""
|
||||
tree, _, _ = parse_formula("=[1, 2, 3]", ALLOWED)
|
||||
result = evaluate_ast(tree, {}, {})
|
||||
assert result == [1, 2, 3]
|
||||
|
||||
|
||||
def test_tuple_literal_evaluated() -> None:
|
||||
"""Tuple literals (1, 2, 3) should be evaluated as Python tuples."""
|
||||
tree, _, _ = parse_formula("=(1, 2, 3)", ALLOWED)
|
||||
result = evaluate_ast(tree, {}, {})
|
||||
assert result == (1, 2, 3)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# U3: Cross-table references {rel.target}
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_parse_cross_table_reference() -> None:
|
||||
"""{rel_field.target_field} is parsed into a single Name node (KTD-2)."""
|
||||
tree, field_mapping, cross_table_mapping = parse_formula("={f_rel.f_target}", ALLOWED)
|
||||
# Single-field mapping should be empty
|
||||
assert field_mapping == {}
|
||||
# Cross-table mapping should have one entry
|
||||
assert len(cross_table_mapping) == 1
|
||||
safe_name, (rel_id, target_id) = next(iter(cross_table_mapping.items()))
|
||||
assert rel_id == "f_rel"
|
||||
assert target_id == "f_target"
|
||||
assert safe_name.startswith("_ct_")
|
||||
assert "__" in safe_name # separator between rel and target
|
||||
|
||||
|
||||
def test_parse_cross_table_with_aggregate() -> None:
|
||||
"""=SUM({rel.target}) — cross-table ref inside an aggregate function."""
|
||||
tree, _, cross_table_mapping = parse_formula("=SUM({f_rel.f_target})", ALLOWED)
|
||||
assert len(cross_table_mapping) == 1
|
||||
safe_name, (rel_id, target_id) = next(iter(cross_table_mapping.items()))
|
||||
assert rel_id == "f_rel"
|
||||
assert target_id == "f_target"
|
||||
# Should evaluate correctly when given a list of values
|
||||
result = evaluate_ast(tree, {safe_name: [1, 2, 3]}, {"SUM": sum})
|
||||
assert result == 6
|
||||
|
||||
|
||||
def test_parse_mixed_single_and_cross_table_refs() -> None:
|
||||
"""={f_local} + SUM({f_rel.f_target}) — mixed row + cross-table aggregate."""
|
||||
tree, field_mapping, cross_table_mapping = parse_formula(
|
||||
"={f_local} + SUM({f_rel.f_target})", ALLOWED
|
||||
)
|
||||
assert "f_local" in field_mapping.values()
|
||||
assert len(cross_table_mapping) == 1
|
||||
safe_local = next(k for k, v in field_mapping.items() if v == "f_local")
|
||||
safe_cross = next(iter(cross_table_mapping.keys()))
|
||||
result = evaluate_ast(
|
||||
tree, {safe_local: 10, safe_cross: [1, 2, 3]}, {"SUM": sum}
|
||||
)
|
||||
assert result == 16
|
||||
|
||||
|
||||
def test_parse_cross_table_uuid_field_ids() -> None:
|
||||
"""Cross-table refs work with UUID-style field IDs containing hyphens."""
|
||||
rel_id = "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
|
||||
target_id = "11111111-2222-3333-4444-555555555555"
|
||||
tree, _, cross_table_mapping = parse_formula(f"={{{rel_id}.{target_id}}}", ALLOWED)
|
||||
assert len(cross_table_mapping) == 1
|
||||
_, (parsed_rel, parsed_target) = next(iter(cross_table_mapping.items()))
|
||||
assert parsed_rel == rel_id
|
||||
assert parsed_target == target_id
|
||||
|
||||
|
||||
def test_cross_table_ref_does_not_match_single_field() -> None:
|
||||
"""{rel.target} must not be partially matched by _FIELD_REF_RE."""
|
||||
# The rel part should not leak into field_mapping
|
||||
tree, field_mapping, cross_table_mapping = parse_formula("={f_rel.f_target}", ALLOWED)
|
||||
assert field_mapping == {}
|
||||
assert len(cross_table_mapping) == 1
|
||||
|
|
|
|||
|
|
@ -0,0 +1,102 @@
|
|||
"""Tests for aggregate and math formula functions (R1)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from agentkit.bitable.formula.functions.aggregate import (
|
||||
AGGREGATE_FUNCTIONS_DICT,
|
||||
AGGREGATE_FUNCTIONS_LIST,
|
||||
)
|
||||
|
||||
|
||||
def test_registry_has_12_functions() -> None:
|
||||
assert len(AGGREGATE_FUNCTIONS_DICT) == 12
|
||||
|
||||
|
||||
def test_aggregate_set_has_10_functions() -> None:
|
||||
assert len(AGGREGATE_FUNCTIONS_LIST) == 10
|
||||
assert "SUM" in AGGREGATE_FUNCTIONS_LIST
|
||||
assert "SUMIF" in AGGREGATE_FUNCTIONS_LIST
|
||||
assert "ABS" not in AGGREGATE_FUNCTIONS_LIST # ABS is scalar, not aggregate
|
||||
assert "ROUND" not in AGGREGATE_FUNCTIONS_LIST
|
||||
|
||||
|
||||
# ── migrated aggregate functions ────────────────────────
|
||||
|
||||
|
||||
def test_sum() -> None:
|
||||
assert AGGREGATE_FUNCTIONS_DICT["SUM"]([1, 2, 3, 4, 5]) == 15
|
||||
assert AGGREGATE_FUNCTIONS_DICT["SUM"]([1, None, "", "3"]) == 4.0
|
||||
|
||||
|
||||
def test_avg() -> None:
|
||||
assert AGGREGATE_FUNCTIONS_DICT["AVG"]([1, 2, 3, 4, 5]) == 3.0
|
||||
assert AGGREGATE_FUNCTIONS_DICT["AVG"]([]) == 0.0
|
||||
|
||||
|
||||
def test_count() -> None:
|
||||
assert AGGREGATE_FUNCTIONS_DICT["COUNT"]([1, 2, None, "", "x"]) == 3
|
||||
|
||||
|
||||
def test_min() -> None:
|
||||
assert AGGREGATE_FUNCTIONS_DICT["MIN"]([3, 1, 4, 1, 5]) == 1.0
|
||||
assert AGGREGATE_FUNCTIONS_DICT["MIN"]([]) == 0
|
||||
|
||||
|
||||
def test_max() -> None:
|
||||
assert AGGREGATE_FUNCTIONS_DICT["MAX"]([3, 1, 4, 1, 5]) == 5.0
|
||||
assert AGGREGATE_FUNCTIONS_DICT["MAX"]([]) == 0
|
||||
|
||||
|
||||
# ── migrated scalar math ────────────────────────────────
|
||||
|
||||
|
||||
def test_abs() -> None:
|
||||
assert AGGREGATE_FUNCTIONS_DICT["ABS"](-5) == 5.0
|
||||
assert AGGREGATE_FUNCTIONS_DICT["ABS"](5) == 5.0
|
||||
assert AGGREGATE_FUNCTIONS_DICT["ABS"](None) is None
|
||||
|
||||
|
||||
def test_round() -> None:
|
||||
assert AGGREGATE_FUNCTIONS_DICT["ROUND"](3.14159, 2) == 3.14
|
||||
assert AGGREGATE_FUNCTIONS_DICT["ROUND"](3.14159) == 3.0
|
||||
assert AGGREGATE_FUNCTIONS_DICT["ROUND"](None) is None
|
||||
|
||||
|
||||
# ── conditional aggregates ──────────────────────────────
|
||||
|
||||
|
||||
def test_sumif_equality() -> None:
|
||||
assert AGGREGATE_FUNCTIONS_DICT["SUMIF"]([1, 2, 3, 4, 5], 3) == 3.0
|
||||
|
||||
|
||||
def test_sumif_greater_than() -> None:
|
||||
assert AGGREGATE_FUNCTIONS_DICT["SUMIF"]([1, 2, 3, 4, 5], ">3") == 9.0 # 4+5
|
||||
|
||||
|
||||
def test_countif_greater_than() -> None:
|
||||
assert AGGREGATE_FUNCTIONS_DICT["COUNTIF"]([1, 2, 3, 4, 5], ">3") == 2
|
||||
|
||||
|
||||
def test_averageif() -> None:
|
||||
assert AGGREGATE_FUNCTIONS_DICT["AVERAGEIF"]([1, 2, 3, 4, 5], ">=3") == 4.0 # (3+4+5)/3
|
||||
|
||||
|
||||
def test_sumifs() -> None:
|
||||
values = [10, 20, 30, 40]
|
||||
criteria_range = [1, 2, 1, 2]
|
||||
# Sum values where criteria_range == 2 → 20 + 40 = 60
|
||||
assert AGGREGATE_FUNCTIONS_DICT["SUMIFS"](values, criteria_range, 2) == 60.0
|
||||
|
||||
|
||||
def test_countifs() -> None:
|
||||
values = [10, 20, 30, 40]
|
||||
criteria_range = [1, 2, 1, 2]
|
||||
# Count values where criteria_range == 1 → 2
|
||||
assert AGGREGATE_FUNCTIONS_DICT["COUNTIFS"](values, criteria_range, 1) == 2
|
||||
|
||||
|
||||
def test_sumifs_length_mismatch() -> None:
|
||||
import pytest
|
||||
|
||||
with pytest.raises(ValueError, match="equal length"):
|
||||
AGGREGATE_FUNCTIONS_DICT["SUMIFS"]([1, 2, 3], [1, 2], 1)
|
||||
|
|
@ -0,0 +1,126 @@
|
|||
"""Tests for datetime formula functions (R1)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from agentkit.bitable.formula.functions.datetime import (
|
||||
DATETIME_FUNCTIONS,
|
||||
_date,
|
||||
_dateadd,
|
||||
_datedif,
|
||||
_day,
|
||||
_eomonth,
|
||||
_hour,
|
||||
_minute,
|
||||
_month,
|
||||
_networkdays,
|
||||
_now,
|
||||
_second,
|
||||
_today,
|
||||
_weekday,
|
||||
_year,
|
||||
)
|
||||
|
||||
|
||||
def test_registry_has_15_functions() -> None:
|
||||
assert len(DATETIME_FUNCTIONS) == 15
|
||||
assert "DATE" in DATETIME_FUNCTIONS
|
||||
assert "NETWORKDAYS" in DATETIME_FUNCTIONS
|
||||
|
||||
|
||||
def test_date_constructor() -> None:
|
||||
result = _date(2026, 7, 6)
|
||||
assert result == "2026-07-06T00:00:00"
|
||||
|
||||
|
||||
def test_year_month_day() -> None:
|
||||
d = _date(2026, 7, 6)
|
||||
assert _year(d) == 2026
|
||||
assert _month(d) == 7
|
||||
assert _day(d) == 6
|
||||
|
||||
|
||||
def test_hour_minute_second() -> None:
|
||||
dt = "2026-07-06T14:30:45"
|
||||
assert _hour(dt) == 14
|
||||
assert _minute(dt) == 30
|
||||
assert _second(dt) == 45
|
||||
|
||||
|
||||
def test_weekday() -> None:
|
||||
# 2026-07-06 is a Monday → ISO weekday 1
|
||||
assert _weekday("2026-07-06") == 1
|
||||
# 2026-07-12 is a Sunday → ISO weekday 7
|
||||
assert _weekday("2026-07-12") == 7
|
||||
|
||||
|
||||
def test_datedif_days() -> None:
|
||||
assert _datedif("2026-07-01", "2026-07-06", "days") == 5
|
||||
|
||||
|
||||
def test_datedif_months() -> None:
|
||||
assert _datedif("2026-01-15", "2026-07-15", "months") == 6
|
||||
|
||||
|
||||
def test_datedif_years() -> None:
|
||||
assert _datedif("2020-01-01", "2026-01-01", "years") == 6
|
||||
|
||||
|
||||
def test_dateadd_days() -> None:
|
||||
assert _dateadd("2026-07-06", 5, "days") == "2026-07-11T00:00:00"
|
||||
|
||||
|
||||
def test_dateadd_months() -> None:
|
||||
assert _dateadd("2026-01-15", 1, "months") == "2026-02-15T00:00:00"
|
||||
|
||||
|
||||
def test_dateadd_months_clamps_day() -> None:
|
||||
# Jan 31 + 1 month → Feb 28 (2026 is not a leap year)
|
||||
assert _dateadd("2026-01-31", 1, "months") == "2026-02-28T00:00:00"
|
||||
|
||||
|
||||
def test_networkdays() -> None:
|
||||
# 2026-07-06 (Mon) to 2026-07-10 (Fri) = 5 working days
|
||||
assert _networkdays("2026-07-06", "2026-07-10") == 5
|
||||
|
||||
|
||||
def test_networkdays_includes_weekend() -> None:
|
||||
# 2026-07-06 (Mon) to 2026-07-12 (Sun) = 5 working days (weekend excluded)
|
||||
assert _networkdays("2026-07-06", "2026-07-12") == 5
|
||||
|
||||
|
||||
def test_networkdays_reversed_args() -> None:
|
||||
# Should swap if start > end
|
||||
assert _networkdays("2026-07-10", "2026-07-06") == 5
|
||||
|
||||
|
||||
def test_eomonth() -> None:
|
||||
# End of July 2026
|
||||
assert _eomonth("2026-07-06", 0) == "2026-07-31T00:00:00"
|
||||
|
||||
|
||||
def test_eomonth_next_month() -> None:
|
||||
# End of August 2026
|
||||
assert _eomonth("2026-07-06", 1) == "2026-08-31T00:00:00"
|
||||
|
||||
|
||||
def test_today_returns_iso_string() -> None:
|
||||
result = _today()
|
||||
assert isinstance(result, str)
|
||||
assert "T00:00:00" in result
|
||||
|
||||
|
||||
def test_now_returns_iso_string() -> None:
|
||||
result = _now()
|
||||
assert isinstance(result, str)
|
||||
assert "T" in result
|
||||
|
||||
|
||||
def test_none_input_returns_none_for_accessors() -> None:
|
||||
assert _year(None) is None
|
||||
assert _month(None) is None
|
||||
assert _day(None) is None
|
||||
|
||||
|
||||
def test_empty_string_returns_none_for_accessors() -> None:
|
||||
assert _year("") is None
|
||||
assert _datedif("", "2026-07-06") is None
|
||||
|
|
@ -0,0 +1,141 @@
|
|||
"""Tests for logical formula functions (R1)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from agentkit.bitable.formula.functions.logical import LOGICAL_FUNCTIONS
|
||||
|
||||
|
||||
def test_registry_has_15_functions() -> None:
|
||||
assert len(LOGICAL_FUNCTIONS) == 15
|
||||
expected = {
|
||||
"IF", "IFS", "SWITCH", "IFERROR", "COALESCE", "AND", "OR", "NOT",
|
||||
"ISBLANK", "ISNUMBER", "ISTEXT", "ISDATE", "TRUE", "FALSE", "NULL",
|
||||
}
|
||||
assert set(LOGICAL_FUNCTIONS.keys()) == expected
|
||||
|
||||
|
||||
# ── IF (migrated) ───────────────────────────────────────
|
||||
|
||||
|
||||
def test_if_true() -> None:
|
||||
assert LOGICAL_FUNCTIONS["IF"](1, "yes", "no") == "yes"
|
||||
|
||||
|
||||
def test_if_false() -> None:
|
||||
assert LOGICAL_FUNCTIONS["IF"](0, "yes", "no") == "no"
|
||||
|
||||
|
||||
def test_if_default_false() -> None:
|
||||
assert LOGICAL_FUNCTIONS["IF"](0, "yes") is None
|
||||
|
||||
|
||||
# ── IFS ─────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_ifs_first_match() -> None:
|
||||
assert LOGICAL_FUNCTIONS["IFS"](1, "a", 0, "b", 1, "c") == "a"
|
||||
|
||||
|
||||
def test_ifs_second_match() -> None:
|
||||
assert LOGICAL_FUNCTIONS["IFS"](0, "a", 1, "b", 0, "c") == "b"
|
||||
|
||||
|
||||
def test_ifs_no_match() -> None:
|
||||
assert LOGICAL_FUNCTIONS["IFS"](0, "a", 0, "b") is None
|
||||
|
||||
|
||||
# ── SWITCH ──────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_switch_match() -> None:
|
||||
assert LOGICAL_FUNCTIONS["SWITCH"]("b", "a", 1, "b", 2, "c", 3) == 2
|
||||
|
||||
|
||||
def test_switch_default() -> None:
|
||||
assert LOGICAL_FUNCTIONS["SWITCH"]("x", "a", 1, "b", 2, "default") == "default"
|
||||
|
||||
|
||||
def test_switch_no_match_no_default() -> None:
|
||||
assert LOGICAL_FUNCTIONS["SWITCH"]("x", "a", 1, "b", 2) is None
|
||||
|
||||
|
||||
# ── IFERROR ─────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_iferror_passes_non_error() -> None:
|
||||
assert LOGICAL_FUNCTIONS["IFERROR"](42, "default") == 42
|
||||
|
||||
|
||||
def test_iferror_returns_default_for_error() -> None:
|
||||
error_dict = {"__error": "division by zero"}
|
||||
assert LOGICAL_FUNCTIONS["IFERROR"](error_dict, "default") == "default"
|
||||
|
||||
|
||||
# ── COALESCE ────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_coalesce_first_non_empty() -> None:
|
||||
assert LOGICAL_FUNCTIONS["COALESCE"]("a", "b", "c") == "a"
|
||||
|
||||
|
||||
def test_coalesce_skips_none_and_empty() -> None:
|
||||
assert LOGICAL_FUNCTIONS["COALESCE"](None, "", "c") == "c"
|
||||
|
||||
|
||||
def test_coalesce_all_empty() -> None:
|
||||
assert LOGICAL_FUNCTIONS["COALESCE"](None, "", None) is None
|
||||
|
||||
|
||||
# ── Boolean operators ───────────────────────────────────
|
||||
|
||||
|
||||
def test_and() -> None:
|
||||
assert LOGICAL_FUNCTIONS["AND"](1, 1, 1) is True
|
||||
assert LOGICAL_FUNCTIONS["AND"](1, 0, 1) is False
|
||||
|
||||
|
||||
def test_or() -> None:
|
||||
assert LOGICAL_FUNCTIONS["OR"](0, 0, 1) is True
|
||||
assert LOGICAL_FUNCTIONS["OR"](0, 0, 0) is False
|
||||
|
||||
|
||||
def test_not() -> None:
|
||||
assert LOGICAL_FUNCTIONS["NOT"](1) is False
|
||||
assert LOGICAL_FUNCTIONS["NOT"](0) is True
|
||||
|
||||
|
||||
# ── Predicates ──────────────────────────────────────────
|
||||
|
||||
|
||||
def test_isblank() -> None:
|
||||
assert LOGICAL_FUNCTIONS["ISBLANK"](None) is True
|
||||
assert LOGICAL_FUNCTIONS["ISBLANK"]("") is True
|
||||
assert LOGICAL_FUNCTIONS["ISBLANK"]("x") is False
|
||||
|
||||
|
||||
def test_isnumber() -> None:
|
||||
assert LOGICAL_FUNCTIONS["ISNUMBER"](42) is True
|
||||
assert LOGICAL_FUNCTIONS["ISNUMBER"](3.14) is True
|
||||
assert LOGICAL_FUNCTIONS["ISNUMBER"]("42") is False
|
||||
assert LOGICAL_FUNCTIONS["ISNUMBER"](True) is False # bool excluded
|
||||
|
||||
|
||||
def test_istext() -> None:
|
||||
assert LOGICAL_FUNCTIONS["ISTEXT"]("hello") is True
|
||||
assert LOGICAL_FUNCTIONS["ISTEXT"](42) is False
|
||||
|
||||
|
||||
def test_isdate() -> None:
|
||||
assert LOGICAL_FUNCTIONS["ISDATE"]("2026-07-06") is True
|
||||
assert LOGICAL_FUNCTIONS["ISDATE"]("2026-07-06T10:30:00") is True
|
||||
assert LOGICAL_FUNCTIONS["ISDATE"]("not a date") is False
|
||||
assert LOGICAL_FUNCTIONS["ISDATE"](42) is False
|
||||
|
||||
|
||||
# ── Constants ───────────────────────────────────────────
|
||||
|
||||
|
||||
def test_true_false_null() -> None:
|
||||
assert LOGICAL_FUNCTIONS["TRUE"]() is True
|
||||
assert LOGICAL_FUNCTIONS["FALSE"]() is False
|
||||
assert LOGICAL_FUNCTIONS["NULL"]() is None
|
||||
|
|
@ -0,0 +1,126 @@
|
|||
"""Tests for text formula functions (R1)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from agentkit.bitable.formula.functions.text import TEXT_FUNCTIONS
|
||||
|
||||
|
||||
# ── Registry ────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_registry_has_17_functions() -> None:
|
||||
assert len(TEXT_FUNCTIONS) == 17
|
||||
expected = {
|
||||
"CONCAT", "LEN", "FIND", "MID", "LEFT", "RIGHT", "REPLACE", "SUBSTITUTE",
|
||||
"UPPER", "LOWER", "TRIM", "REPT", "TEXT", "VALUE", "STARTSWITH", "ENDSWITH",
|
||||
"SPLIT",
|
||||
}
|
||||
assert set(TEXT_FUNCTIONS.keys()) == expected
|
||||
|
||||
|
||||
# ── migrated functions ──────────────────────────────────
|
||||
|
||||
|
||||
def test_concat() -> None:
|
||||
assert TEXT_FUNCTIONS["CONCAT"]("a", "b", "c") == "abc"
|
||||
assert TEXT_FUNCTIONS["CONCAT"]("a", None, "c") == "ac"
|
||||
|
||||
|
||||
def test_len() -> None:
|
||||
assert TEXT_FUNCTIONS["LEN"]("hello") == 5
|
||||
assert TEXT_FUNCTIONS["LEN"](None) == 0
|
||||
assert TEXT_FUNCTIONS["LEN"](123) == 3
|
||||
|
||||
|
||||
# ── find / extract ──────────────────────────────────────
|
||||
|
||||
|
||||
def test_find() -> None:
|
||||
assert TEXT_FUNCTIONS["FIND"]("hello world", "world") == 6
|
||||
assert TEXT_FUNCTIONS["FIND"]("hello", "xyz") == -1
|
||||
assert TEXT_FUNCTIONS["FIND"]("hello", "") == 0
|
||||
|
||||
|
||||
def test_mid() -> None:
|
||||
assert TEXT_FUNCTIONS["MID"]("hello", 1, 3) == "ell"
|
||||
assert TEXT_FUNCTIONS["MID"]("hello", 0, 2) == "he"
|
||||
assert TEXT_FUNCTIONS["MID"]("hello", -1, 2) == "he" # negative clamped
|
||||
|
||||
|
||||
def test_left() -> None:
|
||||
assert TEXT_FUNCTIONS["LEFT"]("hello", 3) == "hel"
|
||||
assert TEXT_FUNCTIONS["LEFT"]("hello") == "h" # default n=1
|
||||
assert TEXT_FUNCTIONS["LEFT"]("hello", 0) == ""
|
||||
|
||||
|
||||
def test_right() -> None:
|
||||
assert TEXT_FUNCTIONS["RIGHT"]("hello", 3) == "llo"
|
||||
assert TEXT_FUNCTIONS["RIGHT"]("hello") == "o" # default n=1
|
||||
|
||||
|
||||
# ── replace / substitute ────────────────────────────────
|
||||
|
||||
|
||||
def test_replace() -> None:
|
||||
# REPLACE("hello", 1, 3, "XYZ") → "h" + "XYZ" + "o" = "hXYZo"
|
||||
# Replaces 3 chars starting at index 1: "ell" → "XYZ"
|
||||
assert TEXT_FUNCTIONS["REPLACE"]("hello", 1, 3, "XYZ") == "hXYZo"
|
||||
|
||||
|
||||
def test_substitute_all() -> None:
|
||||
assert TEXT_FUNCTIONS["SUBSTITUTE"]("a-b-c", "-", "+") == "a+b+c"
|
||||
|
||||
|
||||
def test_substitute_nth() -> None:
|
||||
# Replace only the 2nd occurrence
|
||||
assert TEXT_FUNCTIONS["SUBSTITUTE"]("a-b-c", "-", "+", 2) == "a-b+c"
|
||||
|
||||
|
||||
# ── case / trim ─────────────────────────────────────────
|
||||
|
||||
|
||||
def test_upper_lower() -> None:
|
||||
assert TEXT_FUNCTIONS["UPPER"]("hello") == "HELLO"
|
||||
assert TEXT_FUNCTIONS["LOWER"]("HELLO") == "hello"
|
||||
|
||||
|
||||
def test_trim() -> None:
|
||||
assert TEXT_FUNCTIONS["TRIM"](" hello world ") == "hello world"
|
||||
|
||||
|
||||
# ── repeat / format / convert ───────────────────────────
|
||||
|
||||
|
||||
def test_rept() -> None:
|
||||
assert TEXT_FUNCTIONS["REPT"]("ab", 3) == "ababab"
|
||||
assert TEXT_FUNCTIONS["REPT"]("ab", 0) == ""
|
||||
|
||||
|
||||
def test_text_format() -> None:
|
||||
assert TEXT_FUNCTIONS["TEXT"](3.14159, ".2f") == "3.14"
|
||||
assert TEXT_FUNCTIONS["TEXT"](42) == "42"
|
||||
|
||||
|
||||
def test_value() -> None:
|
||||
assert TEXT_FUNCTIONS["VALUE"]("42") == 42
|
||||
assert TEXT_FUNCTIONS["VALUE"]("3.14") == 3.14
|
||||
assert TEXT_FUNCTIONS["VALUE"]("not a number") is None
|
||||
assert TEXT_FUNCTIONS["VALUE"](None) is None
|
||||
|
||||
|
||||
# ── predicates / split ──────────────────────────────────
|
||||
|
||||
|
||||
def test_startswith() -> None:
|
||||
assert TEXT_FUNCTIONS["STARTSWITH"]("hello world", "hello") is True
|
||||
assert TEXT_FUNCTIONS["STARTSWITH"]("hello world", "world") is False
|
||||
|
||||
|
||||
def test_endswith() -> None:
|
||||
assert TEXT_FUNCTIONS["ENDSWITH"]("hello world", "world") is True
|
||||
assert TEXT_FUNCTIONS["ENDSWITH"]("hello world", "hello") is False
|
||||
|
||||
|
||||
def test_split() -> None:
|
||||
assert TEXT_FUNCTIONS["SPLIT"]("a,b,c", ",") == ["a", "b", "c"]
|
||||
assert TEXT_FUNCTIONS["SPLIT"]("hello", ",") == ["hello"]
|
||||
|
|
@ -12,12 +12,21 @@ import pytest
|
|||
from pydantic import ValidationError
|
||||
|
||||
from agentkit.bitable.models import (
|
||||
AutomationActionType,
|
||||
AutomationLog,
|
||||
AutomationRule,
|
||||
AutomationStatus,
|
||||
AutomationTriggerType,
|
||||
CrossTableDep,
|
||||
CrossTableDepType,
|
||||
Field,
|
||||
FieldOwner,
|
||||
FieldType,
|
||||
Record,
|
||||
RecalcStatus,
|
||||
RecalcTask,
|
||||
RelationLink,
|
||||
RelationType,
|
||||
Table,
|
||||
View,
|
||||
ViewType,
|
||||
|
|
@ -30,7 +39,7 @@ from agentkit.bitable.models import (
|
|||
|
||||
|
||||
def test_field_type_values() -> None:
|
||||
"""FieldType has the 9 supported types with correct string values."""
|
||||
"""FieldType has the 11 supported types with correct string values (V3 adds relation + rollup)."""
|
||||
expected = {
|
||||
"text",
|
||||
"number",
|
||||
|
|
@ -40,7 +49,9 @@ def test_field_type_values() -> None:
|
|||
"attachment",
|
||||
"image",
|
||||
"formula",
|
||||
"relation",
|
||||
"lookup",
|
||||
"rollup",
|
||||
}
|
||||
assert {ft.value for ft in FieldType} == expected
|
||||
|
||||
|
|
@ -61,6 +72,40 @@ def test_recalc_status_lifecycle() -> None:
|
|||
assert {rs.value for rs in RecalcStatus} == {"pending", "calculating", "done", "error"}
|
||||
|
||||
|
||||
def test_relation_type_values() -> None:
|
||||
"""RelationType covers single/bi/multi (R5)."""
|
||||
assert {rt.value for rt in RelationType} == {
|
||||
"one_to_one",
|
||||
"one_to_many",
|
||||
"many_to_many",
|
||||
}
|
||||
|
||||
|
||||
def test_automation_trigger_type_values() -> None:
|
||||
"""AutomationTriggerType covers create/update/delete (R13)."""
|
||||
assert {t.value for t in AutomationTriggerType} == {
|
||||
"record_created",
|
||||
"record_updated",
|
||||
"record_deleted",
|
||||
}
|
||||
|
||||
|
||||
def test_automation_action_type_values() -> None:
|
||||
"""AutomationActionType covers 5 action kinds (R14)."""
|
||||
assert {a.value for a in AutomationActionType} == {
|
||||
"create_record",
|
||||
"update_record",
|
||||
"send_webhook",
|
||||
"send_email",
|
||||
"send_message",
|
||||
}
|
||||
|
||||
|
||||
def test_cross_table_dep_type_values() -> None:
|
||||
"""CrossTableDepType covers formula/lookup/rollup edges (KTD-3)."""
|
||||
assert {d.value for d in CrossTableDepType} == {"formula", "lookup", "rollup"}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Table
|
||||
# ---------------------------------------------------------------------------
|
||||
|
|
@ -301,3 +346,148 @@ def test_table_from_attributes() -> None:
|
|||
table = Table.model_validate(_Row())
|
||||
assert table.id == "t1"
|
||||
assert table.name == "Orders"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# V3 models: RecalcTask.is_cross_table, RelationLink, AutomationRule,
|
||||
# AutomationLog, CrossTableDep
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_recalc_task_is_cross_table_default() -> None:
|
||||
"""RecalcTask.is_cross_table defaults to False (same-table recalc)."""
|
||||
task = RecalcTask(id="q1", table_id="t1", record_id="r1", field_id="f1")
|
||||
assert task.is_cross_table is False
|
||||
|
||||
|
||||
def test_recalc_task_cross_table_flag() -> None:
|
||||
"""RecalcTask can carry is_cross_table=True (U6)."""
|
||||
task = RecalcTask(
|
||||
id="q1",
|
||||
table_id="t1",
|
||||
record_id="r1",
|
||||
field_id="f1",
|
||||
is_cross_table=True,
|
||||
)
|
||||
assert task.is_cross_table is True
|
||||
|
||||
|
||||
def test_relation_link_construction() -> None:
|
||||
"""RelationLink represents a junction-table row (KTD-1)."""
|
||||
link = RelationLink(
|
||||
id="rl1",
|
||||
relation_field_id="f_rel",
|
||||
source_record_id="r_src",
|
||||
target_record_id="r_tgt",
|
||||
)
|
||||
assert link.relation_field_id == "f_rel"
|
||||
assert link.source_record_id == "r_src"
|
||||
assert link.target_record_id == "r_tgt"
|
||||
assert isinstance(link.created_at, datetime)
|
||||
|
||||
|
||||
def test_relation_link_round_trip() -> None:
|
||||
"""RelationLink round-trips through JSON."""
|
||||
link = RelationLink(
|
||||
id="rl1",
|
||||
relation_field_id="f_rel",
|
||||
source_record_id="r_src",
|
||||
target_record_id="r_tgt",
|
||||
)
|
||||
restored = RelationLink.model_validate(link.model_dump(mode="json"))
|
||||
assert restored == link
|
||||
|
||||
|
||||
def test_automation_rule_construction() -> None:
|
||||
"""AutomationRule carries trigger_config + action_config + token."""
|
||||
rule = AutomationRule(
|
||||
id="a1",
|
||||
table_id="t1",
|
||||
name="Notify on done",
|
||||
trigger_config={"type": "record_updated", "field_ids": ["f_status"]},
|
||||
action_config={"type": "send_webhook", "url": "https://example.com"},
|
||||
)
|
||||
assert rule.enabled is True
|
||||
assert rule.webhook_token is None
|
||||
assert rule.trigger_config["type"] == "record_updated"
|
||||
assert rule.action_config["url"] == "https://example.com"
|
||||
|
||||
|
||||
def test_automation_rule_round_trip() -> None:
|
||||
"""AutomationRule round-trips through JSON."""
|
||||
rule = AutomationRule(
|
||||
id="a1",
|
||||
table_id="t1",
|
||||
name="Auto",
|
||||
trigger_config={"type": "record_created"},
|
||||
action_config={"type": "send_message", "channel": "general"},
|
||||
webhook_token="tok_123",
|
||||
)
|
||||
restored = AutomationRule.model_validate(rule.model_dump(mode="json"))
|
||||
assert restored == rule
|
||||
assert restored.webhook_token == "tok_123"
|
||||
|
||||
|
||||
def test_automation_log_construction() -> None:
|
||||
"""AutomationLog carries status + attempt + error_message."""
|
||||
log = AutomationLog(
|
||||
id="log1",
|
||||
automation_id="a1",
|
||||
trigger_record_id="r1",
|
||||
status=AutomationStatus.retrying,
|
||||
attempt=2,
|
||||
error_message="transient",
|
||||
)
|
||||
assert log.status == AutomationStatus.retrying
|
||||
assert log.attempt == 2
|
||||
assert log.error_message == "transient"
|
||||
assert log.completed_at is None
|
||||
|
||||
|
||||
def test_cross_table_dep_construction() -> None:
|
||||
"""CrossTableDep records a source_field → target_field edge (KTD-3)."""
|
||||
dep = CrossTableDep(
|
||||
id="d1",
|
||||
source_table_id="t_src",
|
||||
source_field_id="f_src",
|
||||
target_table_id="t_tgt",
|
||||
target_field_id="f_tgt",
|
||||
dep_type=CrossTableDepType.rollup,
|
||||
)
|
||||
assert dep.dep_type == CrossTableDepType.rollup
|
||||
assert dep.target_table_id == "t_tgt"
|
||||
|
||||
|
||||
def test_field_rollup_config_shape() -> None:
|
||||
"""Rollup field config carries relation_field_id + aggregation (KTD-7)."""
|
||||
field = Field(
|
||||
id="f1",
|
||||
table_id="t1",
|
||||
name="Total Amount",
|
||||
field_type=FieldType.rollup,
|
||||
config={
|
||||
"relation_field_id": "f_rel",
|
||||
"target_field_id": "f_amount",
|
||||
"aggregation": "sum",
|
||||
},
|
||||
)
|
||||
assert field.field_type == FieldType.rollup
|
||||
assert field.config["aggregation"] == "sum"
|
||||
|
||||
|
||||
def test_field_relation_config_shape() -> None:
|
||||
"""Relation field (lookup with relation config) carries relation_type."""
|
||||
field = Field(
|
||||
id="f1",
|
||||
table_id="t1",
|
||||
name="Customer",
|
||||
field_type=FieldType.lookup,
|
||||
config={
|
||||
"relation_type": "many_to_many",
|
||||
"target_table_id": "t_customers",
|
||||
"is_bidirectional": True,
|
||||
"reverse_field_id": "f_reverse",
|
||||
},
|
||||
)
|
||||
assert field.config["relation_type"] == "many_to_many"
|
||||
assert field.config["is_bidirectional"] is True
|
||||
|
|
|
|||
|
|
@ -192,8 +192,9 @@ async def test_crash_recovery_resets_calculating_tasks(
|
|||
tasks = await bitable_service.get_pending_recalc_tasks()
|
||||
assert len(tasks) == 0 # not pending, it's calculating
|
||||
|
||||
# Crash recovery
|
||||
reset_count = await bitable_service.reset_stale_recalc_tasks()
|
||||
# Crash recovery — use threshold=0 to reset all calculating tasks immediately
|
||||
# (simulates worker startup: all calculating tasks are stale when worker was down)
|
||||
reset_count = await bitable_service.reset_stale_recalc_tasks(stale_threshold=0.0)
|
||||
assert reset_count == 1
|
||||
|
||||
# Now it should be pending again
|
||||
|
|
@ -230,11 +231,14 @@ async def test_recalc_deduplication(bitable_service: BitableService) -> None:
|
|||
record = await bitable_service.create_record(table_id=table.id, values={src.id: 10})
|
||||
|
||||
# The create_record already enqueued one task. Enqueue again manually.
|
||||
# With ON CONFLICT DO UPDATE: a duplicate "pending" task is left as-is
|
||||
# (not reset), so the existing task is returned — not None.
|
||||
task2 = await bitable_service.trigger_recalc(table.id, record.id, calc.id)
|
||||
# Should return None (duplicate, ON CONFLICT DO NOTHING)
|
||||
assert task2 is None
|
||||
# task2 may be the existing task (status still "pending") — not None
|
||||
assert task2 is not None
|
||||
assert task2.status == RecalcStatus.pending
|
||||
|
||||
# Only one pending task
|
||||
# Only one pending task (no duplicate created)
|
||||
tasks = await bitable_service.get_pending_recalc_tasks()
|
||||
assert len(tasks) == 1
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,220 @@
|
|||
"""Tests for relation field CRUD service methods (U4: R5, R7, R8).
|
||||
|
||||
These tests require PostgreSQL (skipped when unavailable). They verify:
|
||||
- create_relation_field creates fields with correct config
|
||||
- add/remove_relation_link manages junction table entries (many_to_many)
|
||||
- add/remove_relation_link manages record JSONB (one_to_one/one_to_many)
|
||||
- bidirectional relations create reverse fields
|
||||
- delete_relation_field cascades cleanup
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from agentkit.bitable.models import FieldType, RelationType
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_create_relation_field_one_to_many(bitable_service) -> None:
|
||||
"""create_relation_field creates a relation field with correct config."""
|
||||
table_a = await bitable_service.create_table(name="TableA")
|
||||
table_b = await bitable_service.create_table(name="TableB")
|
||||
|
||||
field = await bitable_service.create_relation_field(
|
||||
table_id=table_a.id,
|
||||
name="Related B",
|
||||
target_table_id=table_b.id,
|
||||
relation_type=RelationType.one_to_many,
|
||||
)
|
||||
|
||||
assert field.field_type == FieldType.relation
|
||||
assert field.config["relation_type"] == "one_to_many"
|
||||
assert field.config["target_table_id"] == table_b.id
|
||||
assert field.config["is_bidirectional"] is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_create_relation_field_bidirectional(bitable_service) -> None:
|
||||
"""Bidirectional relation creates a reverse field on the target table."""
|
||||
table_a = await bitable_service.create_table(name="TableA")
|
||||
table_b = await bitable_service.create_table(name="TableB")
|
||||
|
||||
field = await bitable_service.create_relation_field(
|
||||
table_id=table_a.id,
|
||||
name="Related B",
|
||||
target_table_id=table_b.id,
|
||||
relation_type=RelationType.many_to_many,
|
||||
is_bidirectional=True,
|
||||
reverse_field_name="Reverse A",
|
||||
)
|
||||
|
||||
assert field.config["is_bidirectional"] is True
|
||||
assert "reverse_field_id" in field.config
|
||||
|
||||
reverse_field = await bitable_service.get_field(field.config["reverse_field_id"])
|
||||
assert reverse_field is not None
|
||||
assert reverse_field.name == "Reverse A"
|
||||
assert reverse_field.config["target_table_id"] == table_a.id
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_relation_link_one_to_many(bitable_service) -> None:
|
||||
"""one_to_many relation stores links in record JSONB."""
|
||||
table_a = await bitable_service.create_table(name="TableA")
|
||||
table_b = await bitable_service.create_table(name="TableB")
|
||||
|
||||
field = await bitable_service.create_relation_field(
|
||||
table_id=table_a.id,
|
||||
name="Related B",
|
||||
target_table_id=table_b.id,
|
||||
relation_type=RelationType.one_to_many,
|
||||
)
|
||||
|
||||
rec_a = await bitable_service.create_record(table_a.id, {})
|
||||
rec_b1 = await bitable_service.create_record(table_b.id, {})
|
||||
rec_b2 = await bitable_service.create_record(table_b.id, {})
|
||||
|
||||
# Add links
|
||||
assert await bitable_service.add_relation_link(field.id, rec_a.id, rec_b1.id) is True
|
||||
assert await bitable_service.add_relation_link(field.id, rec_a.id, rec_b2.id) is True
|
||||
|
||||
# Verify
|
||||
related = await bitable_service.list_related_records(field.id, rec_a.id)
|
||||
assert set(related) == {rec_b1.id, rec_b2.id}
|
||||
|
||||
# Idempotent: adding again returns False
|
||||
assert await bitable_service.add_relation_link(field.id, rec_a.id, rec_b1.id) is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_remove_relation_link_one_to_many(bitable_service) -> None:
|
||||
"""remove_relation_link removes from record JSONB."""
|
||||
table_a = await bitable_service.create_table(name="TableA")
|
||||
table_b = await bitable_service.create_table(name="TableB")
|
||||
|
||||
field = await bitable_service.create_relation_field(
|
||||
table_id=table_a.id,
|
||||
name="Related B",
|
||||
target_table_id=table_b.id,
|
||||
relation_type=RelationType.one_to_many,
|
||||
)
|
||||
|
||||
rec_a = await bitable_service.create_record(table_a.id, {})
|
||||
rec_b = await bitable_service.create_record(table_b.id, {})
|
||||
|
||||
await bitable_service.add_relation_link(field.id, rec_a.id, rec_b.id)
|
||||
assert await bitable_service.remove_relation_link(field.id, rec_a.id, rec_b.id) is True
|
||||
assert await bitable_service.list_related_records(field.id, rec_a.id) == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_relation_link_many_to_many(bitable_service) -> None:
|
||||
"""many_to_many relation uses junction table."""
|
||||
table_a = await bitable_service.create_table(name="TableA")
|
||||
table_b = await bitable_service.create_table(name="TableB")
|
||||
|
||||
field = await bitable_service.create_relation_field(
|
||||
table_id=table_a.id,
|
||||
name="Tags",
|
||||
target_table_id=table_b.id,
|
||||
relation_type=RelationType.many_to_many,
|
||||
)
|
||||
|
||||
rec_a = await bitable_service.create_record(table_a.id, {})
|
||||
rec_b1 = await bitable_service.create_record(table_b.id, {})
|
||||
rec_b2 = await bitable_service.create_record(table_b.id, {})
|
||||
|
||||
assert await bitable_service.add_relation_link(field.id, rec_a.id, rec_b1.id) is True
|
||||
assert await bitable_service.add_relation_link(field.id, rec_a.id, rec_b2.id) is True
|
||||
|
||||
related = await bitable_service.list_related_records(field.id, rec_a.id)
|
||||
assert set(related) == {rec_b1.id, rec_b2.id}
|
||||
|
||||
# Reverse lookup
|
||||
reverse = await bitable_service.list_related_records(field.id, rec_b1.id, reverse=True)
|
||||
assert rec_a.id in reverse
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_relation_field_cascades(bitable_service) -> None:
|
||||
"""delete_relation_field removes junction entries and record values."""
|
||||
table_a = await bitable_service.create_table(name="TableA")
|
||||
table_b = await bitable_service.create_table(name="TableB")
|
||||
|
||||
field = await bitable_service.create_relation_field(
|
||||
table_id=table_a.id,
|
||||
name="Related B",
|
||||
target_table_id=table_b.id,
|
||||
relation_type=RelationType.many_to_many,
|
||||
)
|
||||
|
||||
rec_a = await bitable_service.create_record(table_a.id, {})
|
||||
rec_b = await bitable_service.create_record(table_b.id, {})
|
||||
await bitable_service.add_relation_link(field.id, rec_a.id, rec_b.id)
|
||||
|
||||
assert await bitable_service.delete_relation_field(field.id) is True
|
||||
assert await bitable_service.get_field(field.id) is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_relation_link_non_relation_field_raises(bitable_service) -> None:
|
||||
"""add_relation_link on a non-relation field raises ValueError."""
|
||||
table_a = await bitable_service.create_table(name="TableA")
|
||||
field = await bitable_service.create_field(
|
||||
table_id=table_a.id, name="text_field", field_type=FieldType.text
|
||||
)
|
||||
with pytest.raises(ValueError, match="not a relation field"):
|
||||
await bitable_service.add_relation_link(field.id, "rec1", "rec2")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_bidirectional_relation_link_cleanup(bitable_service) -> None:
|
||||
"""remove_relation_link cleans up the reverse field on the target record.
|
||||
|
||||
For bidirectional relations, add_relation_link updates BOTH sides:
|
||||
- Forward: source.values[forward_field] = [target]
|
||||
- Reverse: target.values[reverse_field] = [source]
|
||||
|
||||
remove_relation_link must mirror this — _cleanup_reverse_link removes
|
||||
source from target.values[reverse_field]. Without this, the reverse
|
||||
field retains stale references (asymmetric association).
|
||||
"""
|
||||
table_a = await bitable_service.create_table(name="TableA")
|
||||
table_b = await bitable_service.create_table(name="TableB")
|
||||
|
||||
field = await bitable_service.create_relation_field(
|
||||
table_id=table_a.id,
|
||||
name="Related B",
|
||||
target_table_id=table_b.id,
|
||||
relation_type=RelationType.one_to_many,
|
||||
is_bidirectional=True,
|
||||
reverse_field_name="Reverse A",
|
||||
)
|
||||
reverse_field_id = field.config["reverse_field_id"]
|
||||
|
||||
rec_a = await bitable_service.create_record(table_a.id, {})
|
||||
rec_b = await bitable_service.create_record(table_b.id, {})
|
||||
|
||||
# Add link: forward (A→B) and reverse (B→A) should both be populated
|
||||
await bitable_service.add_relation_link(field.id, rec_a.id, rec_b.id)
|
||||
|
||||
rec_a_after = await bitable_service.get_record(rec_a.id)
|
||||
rec_b_after = await bitable_service.get_record(rec_b.id)
|
||||
assert rec_a_after.values.get(field.id) == [rec_b.id]
|
||||
assert rec_b_after.values.get(reverse_field_id) == [rec_a.id]
|
||||
|
||||
# Remove link: both sides should be cleaned up
|
||||
assert await bitable_service.remove_relation_link(field.id, rec_a.id, rec_b.id) is True
|
||||
|
||||
rec_a_final = await bitable_service.get_record(rec_a.id)
|
||||
rec_b_final = await bitable_service.get_record(rec_b.id)
|
||||
# Forward field: rec_b removed from list
|
||||
forward_val = rec_a_final.values.get(field.id)
|
||||
assert forward_val == [] or forward_val is None
|
||||
# Reverse field: rec_a removed from list (this is the key assertion —
|
||||
# without _cleanup_reverse_link, rec_a would still be here)
|
||||
reverse_val = rec_b_final.values.get(reverse_field_id)
|
||||
assert reverse_val == [] or reverse_val is None, (
|
||||
f"Reverse field not cleaned up, still contains: {reverse_val}"
|
||||
)
|
||||
Loading…
Reference in New Issue