87 lines
12 KiB
Markdown
87 lines
12 KiB
Markdown
# Concepts
|
|
|
|
Shared domain vocabulary for this project — entities, named processes, and status concepts with project-specific meaning. Seeded with core domain vocabulary, then accretes as ce-compound and ce-compound-refresh process learnings; direct edits are fine. Glossary only, not a spec or catch-all.
|
|
|
|
## Bitable
|
|
|
|
### Bitable
|
|
A companion service providing multi-dimensional table storage (structured records with typed fields, views, and formula columns) that runs alongside the main AgentKit server. Owns its own database schema, isolated from core tables, and exposes a REST API boundary that all callers — agents and end-users alike — go through. Agents authenticate with an internal service token; end-users authenticate via JWT.
|
|
|
|
### BitableFile
|
|
The top-level container that groups related data tables. A multi-dimensional table file (多维表格文件) holds zero or more tables and owns its own metadata (name, icon, description, owner). Tables reference their parent file via `file_id`. The file is the unit of ownership sharing and the top of the `文件 → 数据表 → 字段/记录` navigation hierarchy, mirroring the Feishu Bitable App and twenty Object container pattern.
|
|
|
|
### Field Ownership
|
|
The model controlling which actor may modify a field's definition or values. Every field has an owner of either the agent or the user. Agent-owned fields are written by the ingestion tooling during data import; user-owned fields are edited by humans in the UI. Upsert operations update only agent-owned fields — user-owned fields are never overwritten by agent writes, so human edits survive re-ingestion.
|
|
|
|
### Recalc
|
|
The background process that recomputes formula field values after source data changes. When a record's data columns are written, a recalc task is enqueued; a worker claims tasks atomically using row-level locking so concurrent workers partition the queue without coordination, evaluates the formula DAG in topological order, and writes results back. A reaper resets stale in-progress tasks back to pending for crash recovery.
|
|
|
|
## Expert Orchestration
|
|
|
|
### Disclosure Level
|
|
The mode controlling how much of a skill's prompt enters the agent context at load time. Full mode injects the complete skill prompt; summary mode injects only name and description. The default is full so that skills predating the field keep their original behavior — only skills that explicitly opt into summary mode are degraded.
|
|
|
|
### State Offloading
|
|
The pattern of moving a phase's full output to a shared workspace while keeping only a short summary and a reference key in memory. Prevents unbounded memory growth across long pipelines. Consumers that need the full content resolve it through the reference key, not by reading the in-memory summary.
|
|
|
|
### Pipeline Checkpoint
|
|
A crash-recovery snapshot of a pipeline's progress, persisted per phase. On resume, completed phases are skipped, failed phases are preserved as failed, and pending phases execute. Checkpoints are cleared after successful completion; failed runs retain them for retry.
|
|
|
|
### Debate Phase
|
|
A phase dynamically inserted into a team plan when divergence is detected between expert outputs. Subject to a per-pipeline limit that persists across resume — the counter is rebuilt from the restored plan, not reset to zero.
|
|
|
|
### Resume
|
|
The act of rebuilding a crashed pipeline's runtime state from persisted checkpoints. Restores completed and failed phase statuses, rebuilds runtime counters, and re-persists any dynamically inserted phases so the restored plan matches what was executing at crash time.
|
|
|
|
### Verify Re-injection
|
|
The feedback loop triggered when a verification check fails after a final answer is produced. Errors are injected as a new user message into the conversation and the ReAct loop continues for one retry; a second verification failure interrupts execution and returns the error to the user with the verify log. Bounded to one retry to balance auto-correction against token cost, rather than looping until `max_steps`.
|
|
|
|
### Three-tier Degradation Chain
|
|
The agent-level fallback sequence when the primary agent fails: main agent → Recovery tier (reuses `ReflexionEngine` for Evaluate→Reflect→Retry) → Emergency tier (rule-based fallback returning a structured error with suggestions). Each tier is independently configurable; the Recovery tier avoids new infrastructure by reusing the existing reflection engine, and the Emergency tier replaces the previous static-text fallback with actionable error structure.
|
|
|
|
### ReAct Streaming Contract
|
|
The protocol `ReActEngine.execute_stream()` yields to consumers: first zero or more `token` events whose `data.content` are incremental content fragments, then exactly one `final_answer` event whose `data.output` is the concatenation of all token fragments (the complete text). The two events carry the same content — token is the增量 view, final_answer is the聚合 view. Consumers must pick one accumulation strategy (append tokens, or wait for final_answer) and cannot mix both without producing doubled output. When `execute_stream()` is wrapped from a sync `execute()` via `_wrap_sync_as_stream`, no token events are emitted and final_answer's output is the sole content carrier.
|
|
|
|
### Streaming Milestone
|
|
A chat message that progresses through `streaming → completed | error` states as WebSocket events arrive, used to surface long-running expert team operations (expert results, team synthesis) to the user as live progress indicators rather than blocking until completion.
|
|
|
|
A streaming milestone is opened by a chunk event (`expert_result_chunk` / `team_synthesis_chunk`) and must be finalized by a terminal event of the same type family (`expert_result` / `team_synthesis`) carrying a `status` field. If the terminal event never arrives — e.g., the stream is interrupted by cancellation or exception without a cleanup broadcast — the milestone becomes an orphan, spinning forever. Stable identifiers (`synthesis_id`) injected into both chunk and terminal events let the frontend precisely match a terminal event to its open milestone across retries and concurrent teams; positional matching (last streaming milestone) is unreliable in those scenarios and serves only as a backward-compatibility fallback.
|
|
|
|
## Channels & Caching
|
|
|
|
### Per-User Cache Namespace
|
|
A security pattern for LLM response caching where the cache key includes `user_id` so that cached responses are isolated per user. When `per_user_namespace=True`, anonymous requests (`user_id=None`) must be rejected from caching — they cannot be namespaced and would pollute other users' cache blocks. `should_cache()` enforces this by returning `False` when the namespace is on but `user_id` is missing.
|
|
|
|
### Webhook Signature Freshness
|
|
The timestamp validation layered on top of webhook signature verification that bounds the replay window. Computes `abs(now - ts)` against a max-age constant (e.g. 300s) and rejects requests outside the window — the absolute value defends against both historical replays and future-dated requests. Without this, a valid signature alone provides zero replay protection.
|
|
|
|
### Webhook Backpressure
|
|
The pattern of bounding a webhook handler's in-flight background task set with a cap (typically `max_concurrent * 2`) and returning HTTP 429 when exceeded. The 2x margin absorbs short spikes; the 429 forces clients to back off rather than snowballing memory and coroutine exhaustion. The task set is also awaited on app shutdown so in-flight replies are not dropped.
|
|
|
|
## Auth
|
|
|
|
### 3-State Startup
|
|
The auth store's cold-start state machine with values `valid` / `invalid` / `error` (plus the initial `pending`). On app boot, `beginStartup()` runs before `app.mount()`; the router guard `await`s `waitForStartup()` when state is `pending` so no routing decision is made until the probe completes. `invalid` redirects to `/login`; `error` lets navigation proceed (retryable, refresh token retained); `valid` proceeds normally. The distinction matters because a network failure (`error`) must not force a re-login, while a revoked token (`invalid`) must.
|
|
|
|
## Real-Time Fan-Out
|
|
|
|
### Service Broadcast Callback
|
|
The convention for pushing backend state changes to the user's open frontend tabs in real time without coupling domain services to the WebSocket transport. A service accepts an optional async callback at construction; after a successful mutation it best-effort invokes the callback with a typed message envelope. Delivery failure is logged but never rolls back the mutation — the persisted state is the source of truth, the broadcast is a latency optimization. The callback is wired at the composition root (app lifespan) to the portal's per-user fan-out primitive, so the service stays layer-pure. The same callback shape is shared by CRUD services, reminder dispatchers, and sync providers, giving all real-time updates a single exit point.
|
|
|
|
## PLAN_EXEC
|
|
|
|
### Phase State Machine
|
|
The four-phase lifecycle (`PLANNING → BUILDING → VERIFICATION → DELIVERY`) that constrains which tools an agent may call at each step of a PLAN_EXEC run. Each phase has a tool whitelist (e.g., PLANNING allows `search`/`read_file`; BUILDING allows `write_file`); `WILDCARD = "*"` means all tools allowed (used by DELIVERY by default). Transitions are LLM-driven via `AdvancePhaseTool` — the LLM decides when to advance, there is no implicit timer. `auto_advance_after_steps` opts into a step-count trigger as an alternative to explicit `advance_phase` calls. State lives in `ReActEngine.current_phase`; `PhaseState` is an enum with `from_string()` parser.
|
|
|
|
### PhasePolicy
|
|
The dataclass holding the per-phase whitelist (`dict[PhaseState, frozenset[str]]`) and bash command filter (`dict[PhaseState, Callable[[str], bool] | re.Pattern | None]`). Constructed via `default_policy()` (hardcoded KTD5 defaults) or `policy_from_config(plan_exec_cfg)` (YAML-driven). `policy_from_config` returns `None` for empty/disabled config — signaling "opt out, fall back to REACT" (not "use defaults"). `is_bash_command_allowed` accepts either a `Callable` (returns True if dangerous; `ShellTool._is_dangerous` is the default) or a legacy `re.Pattern` (matches dangerous substrings). The `Callable` path closes the regex ceiling — regex missed `:>file`, `dd of=file`, and unknown binaries.
|
|
|
|
### Phase Violation
|
|
The structured event emitted when `_check_phase_permission` blocks a tool call in PLAN_EXEC mode. Two emission paths: (1) the violation is re-injected into the LLM as a tool result so the loop continues (the LLM can switch tools or call `advance_phase`); (2) a `phase_violation` WS event is forwarded to the client for UI feedback (PhaseIndicator component). Violations carry `current_phase`, `tool`, `violation_kind` (`tool_not_allowed` / `bash_command_blocked`), `message`, and `command_preview`. The engine accumulates violations in `_phase_violations` and drains them via `_drain_phase_violations` after each tool dispatch — drains are the caller's responsibility at three sites in `execute_stream`.
|
|
|
|
### AdvancePhaseTool
|
|
The state-transition tool that moves the engine between phases. Calls `engine.advance_phase()` which transitions `current_phase` to the next enum value (PLANNING→BUILDING→VERIFICATION→DELIVERY; DELIVERY is terminal). Returns `{"previous_phase", "current_phase", "message"}`. Bypasses `_check_phase_permission` (always permitted) — phase advancement is not subject to the whitelist it enforces. Known limitation: the default `_loop_threshold=2` fires on the 2nd identical `advance_phase({})` call because all transitions produce the same argument hash — needs exemption from loop detection.
|
|
|
|
### _build_phase_engine
|
|
The chat.py helper that consolidates PLAN_EXEC engine construction for both WS and REST paths. Returns `(engine, tools_with_advance_phase, error_message)`. Returns `(None, None, None)` when `execution_mode != PLAN_EXEC` or `plan_exec.enabled=False` (both signal "fall back to REACT"). Returns `(None, None, error_message)` on policy construction failure. The helper ensures WS and REST paths share identical PhasePolicy + AdvancePhaseTool wiring — previously the REST path returned 501 because it had no construction logic.
|