Commit Graph

51 Commits

Author SHA1 Message Date
chiguyong 5374bc8501 refactor: eliminate routing layer, align with industry best practices
Phase 1 of architecture optimization (U1/U2/U4/U8):

- U1: Rename SimpleRouter to RequestPreprocessor, route() to preprocess()
  Eliminates misleading routing concept; LLM decides autonomously
  in REACT agent loop (matches Codex/Claude Code/Trae pattern)
- U2: Delete CostAwareRouter, HeuristicClassifier, SemanticRouter
  (~700 lines removed). skill_routing.py: 1688 to 220 lines
- U4: PlanExecEngine defaults to ReActStepExecutor, delete _LLMStepExecutor
  (pure LLM calls without tools = no execution capability)
- U8: ReActEngine defaults to ContextCompressor(keep_recent=10)

Supersedes plans 2026-06-15-002/003/004.
New plan: 2026-06-16-006-refactor-architecture-optimization-evolution-plan.md
2026-06-17 10:44:40 +08:00
chiguyong b54213b3c6 fix(review): resolve all P0/P1/P2 findings from code review 2026-06-16 09:08:03 +08:00
chiguyong 16ac592855 feat(gateway): empty response auto-retry with fallback model chain 2026-06-16 08:07:21 +08:00
chiguyong 9caf332e9e fix: ensure agent never returns empty result to user 2026-06-16 08:01:43 +08:00
chiguyong c4257591d4 refactor(router): replace CostAwareRouter with SimpleRouter and prompt-based tool calling 2026-06-16 03:31:05 +08:00
chiguyong 64d62a2b60 feat: autonomous task execution - connect PlanExecEngine + TeamOrchestrator
U1: TeamOrchestrator._execute_phase real execution (Expert.agent.execute)
U2: LLM-based merge strategies (BEST/VOTE/FUSION) with fallback
U3: ReActStepExecutor replacing _LLMStepAgent for tool-enabled steps
U4: SharedWorkspace integration for cross-phase/cross-execution state
U5: GoalPlanner prompt tuning with few-shot and verb pattern matching
U6: Replan-before-fallback in TeamOrchestrator
U7: End-to-end validation tests for multi-step research tasks
U8: WebSocket progress events (step_event_callback + new event types)

Code review fixes: P0 response.strip fix, P1 competitor status check,
milestone real impl, VOTE self-bias fix, confirmation_handler wiring,
ExpertTeam public API, DRY _build_result_summaries, replan tests

Also: geo_server.py refactor (ServerConfig.from_yaml), delete llm_config.yaml
2026-06-15 12:41:32 +08:00
chiguyong 99fe4c99f7 fix: comprehensive code review fixes + WS test stability 2026-06-15 08:17:34 +08:00
chiguyong 7384ecb03e feat: Expert Team Mode — plan-execute collaboration with conversation UI
Implements B+C hybrid Expert Team Mode with ExpertConfig, CollaborationPlan,
TeamOrchestrator, ExpertTeamRouter, HandoffTransport, SharedWorkspace, and
Expert wrapper. Frontend includes ExpertTeamView, ExpertMessage,
PlanVisualization, team store, and WS event handlers.

Code review fixes: sentinel-based close, per-phase retry, name validation,
Vue component integration, teamState dedup, Redis reset, plan reassign,
event_type validation, hmac timing-safe compare, message dedup,
reactive updatePhases, O(1) phase lookup, iterative DFS, bounded Queue.

232 unit tests passing.
2026-06-14 22:20:14 +08:00
chiguyong 0ccef7be5c feat: P0 production hardening — LLM cache, semantic routing, state persistence
U1: LLM Cache Core (exact + semantic match, InMemory + Redis backends)
U2: Cache integration into LLMGateway with CacheConfig
U3: Semantic Router as Layer 1.5 in CostAwareRouter
U4: UsageStore persistence (Redis Hash + InMemory fallback)
U5: CascadeStateStore persistence (Redis INCR + InMemory TTL)
U6: EvolutionStore interface unification (Protocol + PostgreSQL backend)
U7: Configuration integration + E2E tests

Code review fixes:
- P0: date iteration bug (day>=28), semantic router index never built,
      Redis connection leak (per-call → persistent pool)
- P1: cache degradation recovery, semantic_search degradation,
      double miss counting, asyncio.Lock for PG init, LIMIT on queries,
      __import__ anti-pattern → _utcnow()
- P2: InMemory TTL cleanup, embedding preservation on put(),
      data TTL = max(exact_ttl, semantic_ttl)
2026-06-14 15:16:00 +08:00
chiguyong 5ef08a3b30 fix(review): comprehensive P0-P2 code review fixes 2026-06-12 22:18:25 +08:00
chiguyong 2e55aae775 fix(review): address code review findings for speed optimization
- P0: Rename WAL buffer to pending buffer, add crash-loss warning
- P1: Fix keyword substring false matches with word-boundary regex
- P1: Pass connection pool params in _build_llm_config
- P1: Change parallel_tools default to False (safer default)
- P1: Add classifier value validation in CostAwareRouter
- P2: Replace __import__ with proper datetime import
- P2: Add max_buffer_size enforcement in AsyncWriteQueue
2026-06-12 13:21:44 +08:00
chiguyong a36bc3d1c1 feat: optimize chat response speed for sub-1s first token latency
- Add HeuristicClassifier to replace LLM quick_classify with zero-cost
  local heuristic (keyword/length/code-pattern scoring), gated by
  router.classifier config (default: heuristic)
- Add parallel tool execution in ReActEngine via asyncio.gather for
  multiple independent tool_calls, gated by parallel_tools param
- Add AsyncWriteQueue for non-blocking session persistence with WAL
  buffer, gated by async_writes param on SessionManager
- Add httpx.Limits connection pool config to all LLM providers
- Add router config section to ServerConfig and agentkit.yaml
- All optimizations have config switches for safe rollback
2026-06-12 13:15:06 +08:00
chiguyong 55421dd126 fix: get_tools() and get_system_prompt() now read from tool_registry too
Root cause: app.py registers tools via agent._tool_registry.register()
which adds to the ToolRegistry but NOT to agent._tools (which is only
populated by use_tool() from config). Both get_tools() and
get_system_prompt() were reading only _tools, missing all post-init
registered tools. Now both methods merge _tools with
_tool_registry.list_tools().
2026-06-11 22:17:14 +08:00
chiguyong f7225bc91a fix: include available tools in system prompt so LLM knows what it can call
Previously get_system_prompt() only returned identity/instructions but
did not tell the LLM what tools are available. The LLM would therefore
refuse to call tools even when they were registered, saying it had no
tools. Now the system prompt includes a '## 可用工具' section listing
all registered tools with their descriptions and parameters.
2026-06-11 22:00:30 +08:00
chiguyong 32c800d1e4 fix: portal routing + response speed + IME input
1. Portal unified routing: ws_chat now uses CostAwareRouter uniformly
   (handles Layer 0/1/2), replacing direct IntentRouter calls.
   Greeting/chat_mode requests skip IntentRouter LLM call entirely.

2. Response speed: greeting & simple chat now use direct LLM call
   (no ReAct loop), zero-cost Layer 0 detection.

3. IME input fix: use e.isComposing (native browser property)
   instead of compositionstart/end for Enter key detection.

4. Test: fix InMemoryMessageBus.request() parameter name
   timeout -> timeout_seconds.
2026-06-11 21:30:25 +08:00
chiguyong d47f279887 fix: resolve code review issues from deferred improvements
1. InMemoryMessageBus.request(): fix param name (timeout→timeout_seconds) to match ABC
2. InMemoryMessageBus: track consumer tasks, cancel on unsubscribe
3. InMemoryMessageBus: _try_resolve_pending() in queue consumer path
4. evolve_soul(): use "default" category when patterns is empty
5. quick_classify(): use delimiter-based prompt to mitigate injection risk
6. Use asyncio.get_running_loop() instead of deprecated get_event_loop()
2026-06-11 13:49:02 +08:00
chiguyong ec51dbb259 feat: optimize劣势项 — 拍卖开关/审计采样/线程安全/评分锚定
1. 拍卖机制: 已有配置开关(marketplace.auction_enabled), 默认关闭
2. LLM审计采样: 新增 audit_sample_rate (0.0-1.0), 默认1.0, 可降低审计频率
3. AlignmentConfig.from_dict: 忽略未知键, 防止YAML额外字段崩溃
4. 配置热重载线程安全: 用 threading.Event 替代布尔标志, 消除数据竞态
5. Reflexion评分锚定: 添加评分维度(Completeness/Correctness/Clarity)和锚定点
2026-06-11 13:04:36 +08:00
chiguyong cc2cd414c9 fix: resolve all code review issues from cross-validation
1. Critical: Add missing TaskResult import in plan_exec_engine.py
2. Critical: Fix ReWOOEngine param name (max_steps → max_plan_steps)
3. Major: Remove duplicate token counting in reflexion.py
4. Major: LLM audit failure now passes (trusts rule check) instead of failing
5. Major: Fix dict iteration with del using list() copy in lifecycle.py
6. Major: Fix Chinese content tokenization using regex split instead of space split
7. Minor: _is_positive_mention now checks all occurrences, not just the first
2026-06-11 06:22:35 +08:00
chiguyong 79eb8469f9 fix: address remaining code review issues
- AlignmentGuard: direction-aware constraint checking (negation/affirmation detection)
  instead of simple substring matching to reduce false positives
- Reflexion: extract actual token usage from LLM response instead of hardcoded 1
- MemoryTool: protect version/history sections from update_soul modification
- Fix AsyncMock warnings for sync find_best_agent method
2026-06-11 00:14:11 +08:00
chiguyong 5b42487d8a feat(core): add ReWOO, Plan-and-Execute, Reflexion execution engines
Phase A of Multi-Agent Marketplace architecture:
- ReWOOEngine: plan-all-then-execute pattern for parallel data fetch
- PlanExecEngine: adapter wrapping GoalPlanner+PlanExecutor+PipelineReplanner
- ReflexionEngine: ReAct + Evaluate + Reflect + Retry for high-precision tasks
- SkillConfig: extend VALID_EXECUTION_MODES with rewoo/plan_exec/reflexion
- ConfigDrivenAgent: add _handle_rewoo/_handle_plan_exec/_handle_reflexion routes
- 5 professional agent YAML configs with layered model defaults
- 107 unit tests passing
2026-06-10 17:08:48 +08:00
chiguyong 6852dfe892 fix(security,reliability): resolve all P2 findings from code review 2026-06-10 15:05:40 +08:00
chiguyong 658e188939 fix(review): resolve P0/P1 findings from final code review 2026-06-10 09:57:29 +08:00
chiguyong 1d1805753c fix: resolve key P2 findings from code review
- Shell whitelist: use exact binary match instead of startswith
- Shell audit log: use deque(maxlen=10000) to cap memory
- Terminal history: use deque(maxlen) for O(1) eviction
- Path optimizer: cap _pending_paths at 50 entries per task_type
- Pitfall detector: only add tips to matching steps, not all
- Experience store: handle non-numeric _parse_time_window input
- Extract shared is_safe_url() to utils/security.py (DRY)
- Workflow condition evaluator: handle float() ValueError
2026-06-10 09:01:23 +08:00
chiguyong 7874e875af merge: integrate feat/agentkit-phase8-chat-adaptive (chat/gui commands + GUI mode)
Restores agentkit chat, agentkit gui CLI commands, onboarding wizard,
and GUI mode (AGENTKIT_GUI_MODE) with static file serving.
Resolves merge conflicts in orchestrator.py, app.py, tools/__init__.py, shell.py.
2026-06-10 07:44:06 +08:00
chiguyong 9e9f1314f6 fix(security): resolve all P0/P1 findings from code review 2026-06-10 07:12:41 +08:00
chiguyong fd4a811929 feat(phase1): implement core kernel and experience foundation (U1-U5)
- U1: GoalPlanner - structured goal decomposition wrapping _decompose_task()
- U2: PlanExecutor - parallel execution with retry/skip/replace strategies
- U3: PlanChecker - quality gate + review + experience writing
- U4: Skill spec upgrade - dependencies, capabilities, version management
- U5: ExperienceStore - PostgreSQL+pgvector task experience storage

208 new tests passing, fully backward compatible.
2026-06-09 23:57:03 +08:00
chiguyong 45283d31e8 feat(core): integrate MessageBus into Orchestrator and AgentPool (U7)
- Orchestrator accepts optional message_bus parameter; workers publish
  task.progress messages via MessageBus after each subtask execution
- AgentPool accepts optional message_bus; auto-registers agents on
  create and auto-unregisters on remove
- app.py initializes MessageBus from config and injects into AgentPool
- ServerConfig adds bus configuration field
- 5 new tests, all passing
2026-06-08 00:03:40 +08:00
chiguyong 88d8298871 feat(core): add Orchestrator adaptive task decomposition (U5)
- execute_adaptive(): iterative execute→evaluate→re-decompose loop
- OrchestratorConfig: adaptive, max_iterations, quality_threshold
- _evaluate_quality(): LLM-based or rule-based quality scoring (0-1)
- _reexecute_failed(): preserves completed subtask results, retries
  failed ones with improvement feedback injected into input_data
- OrchestrationResult.metadata field for tracking iteration history
- 10 new tests, all passing
2026-06-07 23:50:54 +08:00
chiguyong 7054ac02b6 feat(tools): add AskHumanTool + token streaming in ReAct execute_stream
- AskHumanTool: Human-in-the-Loop tool for Chat mode, pushes questions
  via WebSocket callback and waits for user reply via asyncio.Future
- Token streaming: execute_stream() now uses chat_stream() instead of
  chat(), yielding token-type ReActEvents for each StreamChunk
- _build_response_from_stream() static method constructs LLMResponse
  from accumulated stream data
- Export AskHumanTool from tools/__init__.py
- 12 new tests (7 AskHumanTool + 5 token streaming), all passing
2026-06-07 23:40:43 +08:00
chiguyong b34b06724d fix(agentkit): resolve all P0/P1/P2/P3 issues from code review 2026-06-07 22:05:18 +08:00
chiguyong 286804792d feat(compression): U4 ServerConfig compression field and Agent injection
Add compression config to ServerConfig (following telemetry pattern),
create compressor in create_app, pass through AgentPool to
ConfigDrivenAgent, and inject into ReActEngine.execute() calls.
2026-06-07 18:20:05 +08:00
chiguyong fcb4fb33f3 feat(compression): U3 ReAct engine tool result compression and incremental compress
Extend _build_tool_result_message to accept compressor parameter for
tool output compression. Add _should_compress helper for token budget
checking. Add incremental compression within ReAct loop when
conversation exceeds threshold.
2026-06-07 18:19:53 +08:00
chiguyong ea705b979b feat(compression): U2 HeadroomCompressor with SmartCrusher and CCR cache
Add HeadroomCompressor implementing CompressionStrategy Protocol with
content-type routing (JSON→SmartCrusher, code→CodeCompressor), CCR
reversible compression cache, and graceful degradation when headroom-ai
is not installed.
2026-06-07 18:19:41 +08:00
chiguyong 5d3a5f2bf3 feat(compression): U1 CompressionStrategy Protocol and create_compressor factory
Add runtime-checkable CompressionStrategy Protocol with compress(),
compress_tool_result(), and is_available() methods. Add compress_tool_result
and is_available to existing ContextCompressor. Add create_compressor()
factory function with headroom/summary provider routing and ImportError
fallback.
2026-06-07 18:19:27 +08:00
chiguyong 239009357a feat(telemetry): U7 OpenTelemetry integration with zero-dependency no-op pattern
Add telemetry module with tracing (agent/tool/llm/pipeline_step spans),
metrics (5 histograms/counters), and setup with optional OTLP exporters.
Uses no-op pattern when opentelemetry not installed. GenAI Semantic
Conventions for LLM spans. Integrated into ReactEngine, LLMGateway,
ToolBase, and FastAPI app.
2026-06-07 17:26:21 +08:00
chiguyong 24e501f745 fix(core): U10 Agent status lock timeout and config hot-reload audit
- Added _acquire_status_lock with timeout (30s) to prevent deadlocks
- Added _release_status_lock for safe lock release
- Added config_version tracking on BaseAgent
- Config hot-reload now increments version and propagates to agents
- Audit logging with config version in _on_config_change
2026-06-06 22:52:51 +08:00
chiguyong 23934602c0 feat(core): U4 multi-agent Orchestrator with SharedWorkspace
- Orchestrator: Orchestrator-Worker pattern with LLM-driven task decomposition
- SharedWorkspace: Redis-backed shared state with versioning and distributed locks
- SubTask dependency graph, parallel group building, result aggregation
- 16 tests passing
2026-06-06 22:25:12 +08:00
chiguyong 6e362a8ae7 feat(agentkit): Phase 4 enterprise production upgrade — 12 Implementation Units
Phase A (P0): EpisodicMemory pgvector search+EmbeddingCache, ReAct timeout+CancellationToken, evolution system fix (A/B test+LLMPromptOptimizer+StrategyTuner), AnthropicProvider native Messages API
Phase B (P1): RetryPolicy+CircuitBreaker, chat_stream fallback chain, WebSocket endpoint, SSE stream fix, Evolution+Memory API routes (7 endpoints), embedding cache+Enhanced Search per-KB degradation fix
Phase C (P2): GeminiProvider native generateContent API, Agent state lock+config hot-reload

Tests: 1301 passed, 18 skipped, 0 failed
2026-06-06 21:51:04 +08:00
chiguyong e33dc25ad3 feat(memory): RAG pipeline optimization — 5 Implementation Units
U1: QueryTransformer — LLM/rule-based query rewriting + sub-query decomposition
U2: HttpRAGService enhanced_search() — rerank + compression via /bases/{kb_id}/retrieve
U3: Structured context injection — source attribution headers in RAG results
U4: RetrieveKnowledgeTool — built-in tool for mid-reasoning knowledge retrieval
U5: Configurable retrieval params + per-KB weights + CJK token estimation

Config example:
  memory:
    retrieval:
      top_k: 5
      token_budget: 2000
      context_template: structured
    query_transform:
      enabled: true
      strategy: llm
    semantic:
      search_mode: enhanced
      use_rerank: true
      kb_weights:
        industry-kb-id: 1.2
        enterprise-kb-id: 0.8

Tests: 1037 passed, 18 skipped, 0 failed
2026-06-06 19:27:09 +08:00
chiguyong cd5b39087e feat(memory): add HttpRAGService for config-driven knowledge base integration 2026-06-06 18:36:05 +08:00
chiguyong 0456429beb fix(review): address all 14 P2 advisory findings 2026-06-06 18:20:46 +08:00
chiguyong 8620751864 fix(review): address P0+P1 findings from Tier 2 code review
P0: MemoryRetriever.retrieve score mutation fix
P1: Redis atomic Lua script, deprecated API fix, SQLite WAL mode,
Redis URL masking, UniqueConstraint, TraceRecorder completed flag,
EpisodicMemory recall improvement, LLMReflector sanitization,
A/B test safety, generator cleanup, ContextCompressor guards,
OpenAIEmbedder reuse, Pipeline failure handling, Metrics O(1),
Health check Redis PING, CLI skill loading, CORS config,
API key direct pass-through

Tests: 924 passed, 18 skipped, 0 failed
2026-06-06 17:57:47 +08:00
chiguyong f858d279f3 feat(agentkit): Phase 3 upgrade - persistence, memory, evolution, observability
10 Implementation Units across 3 phases:

Phase A - Infrastructure:
- U1: RedisTaskStore with Redis/memory backend + factory function
- U2: TraceRecorder for execution trace recording
- U3: PersistentEvolutionStore with SQLite backend

Phase B - Core Capabilities:
- U4: MemoryRetriever integration into ReAct engine
- U5: Embedder abstraction + EpisodicMemory vector search
- U6: LLMReflector for LLM-in-the-loop reflection
- U7: SkillPipeline for multi-skill orchestration

Phase C - Enhancement:
- U8: SKILL.md format + progressive disclosure levels
- U9: ContextCompressor + prompt cache rendering
- U10: Structured logging + metrics endpoint + enhanced health check

Tests: 924 passed, 18 skipped, 0 failed
2026-06-06 17:17:45 +08:00
chiguyong acec8ff743 feat(evolution): Phase A - lifecycle hooks + EvolutionConfig
U11: EvolutionMixin integrated into ConfigDrivenAgent lifecycle
  - on_task_complete triggers evolve_after_task
  - on_task_failed records failure patterns
  - Evolution errors never break main task flow

U12: EvolutionConfig added to SkillConfig
  - enabled, reflect_on_failure, auto_apply, min_quality_threshold
  - Backward compatible: defaults to enabled=False

21 new tests passing, no regression.
2026-06-06 12:05:56 +08:00
chiguyong 2844eeb548 feat(streaming): Phase C - LLM streaming + ReAct events + SSE endpoint
U8: StreamChunk protocol + OpenAI chat_stream + Gateway streaming with usage tracking
U9: ReActEvent dataclass + execute_stream() yielding thinking/tool_call/tool_result/final_answer
U10: POST /tasks/stream SSE endpoint + Client SDK stream_task()

15 new tests passing, no regression.
2026-06-06 11:54:17 +08:00
chiguyong 5f1c51cf9a feat(server): Phase B - auth, rate limiting, SSRF protection, handler whitelist
U1: API Key authentication middleware (dev mode skip, health whitelist)
U2: Rate limiting middleware (fixed-window, 60 req/min default)
U3: Callback URL SSRF protection (private IP blocking)
U4: custom_handler module prefix whitelist

65 tests passing. CORS conflict fixed.
2026-06-05 23:37:36 +08:00
chiguyong f87b790c0f feat(agentkit): v2 Phase 1 - ReAct/LLM Gateway/Skill/Server + review fixes
535 unit + 52 integration tests passing. README added.
2026-06-05 23:32:16 +08:00
chiguyong 5a90824c77 feat(core): add ConfigDrivenAgent with YAML-driven agent definition
- AgentConfig: YAML/dict config model with validation
- ConfigDrivenAgent: 3 task modes (llm_generate, tool_call, custom)
- StandaloneRunner: auto-discover YAML configs and build agents
- 25 new tests covering all modes and edge cases
- Total: 56 tests passing
2026-06-04 22:39:25 +08:00
chiguyong 2ddffcdf37 chore: add .gitignore and remove cached files 2026-06-04 22:28:44 +08:00
chiguyong cc3dfd44e3 fix: switch to setuptools for Python 3.14 compatibility 2026-06-04 22:27:06 +08:00