Commit Graph

65 Commits

Author SHA1 Message Date
chiguyong 31bd3b126c feat(phase8): chat adaptive enhancements, pipeline reflection, search tools upgrade
- Enhanced chat CLI with adaptive mode and session management
- Added pipeline reflection and schema extensions
- Upgraded BaiduSearch and WebSearch tools with advanced capabilities
- Expanded server routes for skills and chat
- Added session store enhancements
- New chat module and pipeline reflection support
2026-06-09 23:18:06 +08:00
chiguyong 045fecd4ce feat(tools): add ShellTool + WebSearchTool, memory system, onboarding wizard, chat mode
- ShellTool: safe command execution with allowlist, blocked patterns (regex), timeout, output truncation
- WebSearchTool: multi-backend search with Tavily → Serper → DuckDuckGo Lite fallback
- MemoryTool: agent-callable tool with add/replace/remove/read actions
- MemoryStore/MemoryFile: file-based memory (SOUL.md, USER.md, MEMORY.md, DAILY.md)
- Onboarding wizard: provider selection, API key, model selection, agent personality
- Chat mode: interactive CLI with streaming, memory injection, tool integration
- Add 百炼 Coding Plan provider with 10 models
- 102 unit tests (34 new for ShellTool + WebSearchTool)
2026-06-09 01:06:45 +08:00
chiguyong 9874a4aac0 test: add Phase 8 integration tests for Chat + Adaptive + Multi-Agent (U8)
End-to-end integration tests covering session lifecycle, adaptive pipeline,
multi-agent communication via MessageBus, and config serialization.
2026-06-08 01:17:04 +08:00
chiguyong 45283d31e8 feat(core): integrate MessageBus into Orchestrator and AgentPool (U7)
- Orchestrator accepts optional message_bus parameter; workers publish
  task.progress messages via MessageBus after each subtask execution
- AgentPool accepts optional message_bus; auto-registers agents on
  create and auto-unregisters on remove
- app.py initializes MessageBus from config and injects into AgentPool
- ServerConfig adds bus configuration field
- 5 new tests, all passing
2026-06-08 00:03:40 +08:00
chiguyong 13d6e74099 feat(bus): add MessageBus abstraction layer with InMemory + Redis Streams (U6)
- AgentMessage: message model with sender/recipient/topic/payload/correlation_id
- MessageBus Protocol: publish/subscribe/unsubscribe/request/broadcast/health_check
- InMemoryMessageBus: asyncio.Queue-based implementation for testing
- RedisMessageBus: Redis Streams (XADD/XREADGROUP) implementation with
  consumer groups, message acknowledgment, and dead letter queue
- create_message_bus() factory with graceful Redis→InMemory fallback
- Request-response pattern via correlation_id + asyncio.Future
- 13 new tests, all passing
2026-06-07 23:58:16 +08:00
chiguyong 88d8298871 feat(core): add Orchestrator adaptive task decomposition (U5)
- execute_adaptive(): iterative execute→evaluate→re-decompose loop
- OrchestratorConfig: adaptive, max_iterations, quality_threshold
- _evaluate_quality(): LLM-based or rule-based quality scoring (0-1)
- _reexecute_failed(): preserves completed subtask results, retries
  failed ones with improvement feedback injected into input_data
- OrchestrationResult.metadata field for tracking iteration history
- 10 new tests, all passing
2026-06-07 23:50:54 +08:00
chiguyong 7054ac02b6 feat(tools): add AskHumanTool + token streaming in ReAct execute_stream
- AskHumanTool: Human-in-the-Loop tool for Chat mode, pushes questions
  via WebSocket callback and waits for user reply via asyncio.Future
- Token streaming: execute_stream() now uses chat_stream() instead of
  chat(), yielding token-type ReActEvents for each StreamChunk
- _build_response_from_stream() static method constructs LLMResponse
  from accumulated stream data
- Export AskHumanTool from tools/__init__.py
- 12 new tests (7 AskHumanTool + 5 token streaming), all passing
2026-06-07 23:40:43 +08:00
chiguyong 6013d5189b feat(chat): add Chat API routes with REST + WebSocket bidirectional communication 2026-06-07 22:49:26 +08:00
chiguyong 493187782c feat(session): add Session/Message models and SessionManager with InMemory/Redis stores 2026-06-07 22:43:14 +08:00
chiguyong e4d6efb4bf Merge feat/agentkit-phase7-headroom: Phase 6-7 + all review fixes 2026-06-07 22:05:34 +08:00
chiguyong b34b06724d fix(agentkit): resolve all P0/P1/P2/P3 issues from code review 2026-06-07 22:05:18 +08:00
chiguyong 3645c7a080 docs: mark Phase 7 Headroom integration plan as completed 2026-06-07 18:21:27 +08:00
chiguyong bad66445ff feat(compression): U6 GEO Pipeline compression integration tests and config
Add GEO Pipeline end-to-end compression integration tests with
MockHeadroomCompressor. Add compression configuration section to
llm_config.yaml with headroom and summary mode examples.
2026-06-07 18:20:41 +08:00
chiguyong 9c04362dba feat(compression): U5 HeadroomRetrieveTool for CCR cache retrieval
Add HeadroomRetrieveTool that allows LLM to retrieve original
uncompressed data from CCR cache via Function Calling. Auto-registered
when HeadroomCompressor is active and available.
2026-06-07 18:20:17 +08:00
chiguyong 286804792d feat(compression): U4 ServerConfig compression field and Agent injection
Add compression config to ServerConfig (following telemetry pattern),
create compressor in create_app, pass through AgentPool to
ConfigDrivenAgent, and inject into ReActEngine.execute() calls.
2026-06-07 18:20:05 +08:00
chiguyong fcb4fb33f3 feat(compression): U3 ReAct engine tool result compression and incremental compress
Extend _build_tool_result_message to accept compressor parameter for
tool output compression. Add _should_compress helper for token budget
checking. Add incremental compression within ReAct loop when
conversation exceeds threshold.
2026-06-07 18:19:53 +08:00
chiguyong ea705b979b feat(compression): U2 HeadroomCompressor with SmartCrusher and CCR cache
Add HeadroomCompressor implementing CompressionStrategy Protocol with
content-type routing (JSON→SmartCrusher, code→CodeCompressor), CCR
reversible compression cache, and graceful degradation when headroom-ai
is not installed.
2026-06-07 18:19:41 +08:00
chiguyong 5d3a5f2bf3 feat(compression): U1 CompressionStrategy Protocol and create_compressor factory
Add runtime-checkable CompressionStrategy Protocol with compress(),
compress_tool_result(), and is_available() methods. Add compress_tool_result
and is_available to existing ContextCompressor. Add create_compressor()
factory function with headroom/summary provider routing and ImportError
fallback.
2026-06-07 18:19:27 +08:00
chiguyong 80a505b1c1 docs: mark Phase 6 plan as completed 2026-06-07 17:27:01 +08:00
chiguyong 239009357a feat(telemetry): U7 OpenTelemetry integration with zero-dependency no-op pattern
Add telemetry module with tracing (agent/tool/llm/pipeline_step spans),
metrics (5 histograms/counters), and setup with optional OTLP exporters.
Uses no-op pattern when opentelemetry not installed. GenAI Semantic
Conventions for LLM spans. Integrated into ReactEngine, LLMGateway,
ToolBase, and FastAPI app.
2026-06-07 17:26:21 +08:00
chiguyong 03a5167366 feat(pipeline): U6 step-level retry with exponential backoff and saga compensation
Add StepRetryPolicy with jitter-based exponential backoff, SagaOrchestrator
with LIFO compensation pattern, integrate retry_policy and compensate
fields into PipelineStage/PipelineStep schema, add GEO pipeline
compensation definitions for all 7 steps.
2026-06-07 17:26:07 +08:00
chiguyong 4db637cd4f feat(pipeline): U5 state persistence with Redis hot + PG cold dual-write
Add PipelineStateMemory/Redis/PG backends, PipelineStateManager with
Redis Sorted Set hot state + PostgreSQL JSONB cold persistence.
Integrated into PipelineEngine with state persistence calls at each
step transition.
2026-06-07 17:25:52 +08:00
chiguyong 2e547e345a feat(geo): U4 GEO skill tool binding with BaiduSearch and E2E tests
Add BaiduSearchTool (API mode + scraping fallback), bind tools to
GEO skill YAML configs (baidu_search, web_crawl, schema_extract,
schema_generate), extend geo_full_pipeline with generate_content
and deai steps, add 36 E2E integration tests.
2026-06-07 17:25:37 +08:00
chiguyong 9ec1740047 feat(tools): U3 built-in Python tools - WebCrawl, SchemaExtract, SchemaGenerate
Add WebCrawlTool (Crawl4AI wrapper with graceful degradation),
SchemaExtractTool (extruct-based Schema.org extraction), and
SchemaGenerateTool (JSON-LD generation with optional pydantic-schemaorg
validation). All tools work without optional dependencies.
2026-06-07 17:25:24 +08:00
chiguyong 550d29a139 feat(mcp): U2 MCP config system and MCPManager lifecycle
Add MCPServerConfig dataclass with stdio/streamable_http/sse transport
validation, MCPManager for declarative YAML-driven MCP server lifecycle
(start_all/stop_all), tool discovery and registration. Integrated
into FastAPI lifespan startup/shutdown.
2026-06-07 17:25:07 +08:00
chiguyong 66b9217569 feat(mcp): U1 StdioTransport for subprocess-based MCP communication
Add StdioTransport class supporting stdio JSON-RPC over subprocess
stdin/stdout with asyncio.create_subprocess_exec, pending futures
for request/response matching, and stderr forwarding.
2026-06-07 17:24:52 +08:00
chiguyong 9b6c0230c0 docs: add Phase 6 toolkit plan 2026-06-07 16:21:50 +08:00
chiguyong 11a12fed29 docs: mark Phase 5 plan as completed 2026-06-06 22:53:14 +08:00
chiguyong 24e501f745 fix(core): U10 Agent status lock timeout and config hot-reload audit
- Added _acquire_status_lock with timeout (30s) to prevent deadlocks
- Added _release_status_lock for safe lock release
- Added config_version tracking on BaseAgent
- Config hot-reload now increments version and propagates to agents
- Audit logging with config version in _on_config_change
2026-06-06 22:52:51 +08:00
chiguyong 83cdddd199 feat(evaluation): U9 Ragas evaluation pipeline for RAG quality assessment
- RagasEvaluator: LLM-as-Judge evaluation with ragas lib or built-in fallback
- EvalDatasetBuilder: from traces or dict list
- EvalMetrics: faithfulness, answer_relevancy, context_precision, context_recall
- Built-in heuristic evaluation using keyword overlap and Jaccard similarity
- 13 tests passing
2026-06-06 22:49:27 +08:00
chiguyong 9753a08ac8 feat(llm): U8 Chinese LLM providers - Wenxin, Doubao, Yuanbao
- WenxinProvider: Baidu ERNIE via Qianfan v2 OpenAI-compatible API, AK/SK token auth
- DoubaoProvider: ByteDance Doubao via Volcengine Ark API
- YuanbaoProvider: Tencent Hunyuan via OpenAI-compatible API with enhancement mode
- All inherit from OpenAICompatibleProvider for retry/circuit breaker support
- 16 tests passing
2026-06-06 22:46:53 +08:00
chiguyong 34e083abde feat(evolution): U7 multi-objective fitness and extended strategy space
- MultiObjectiveFitness: weighted scoring, NSGA-II Pareto ranking, crowding distance
- FitnessWeights: configurable accuracy/latency/cost weights with auto-normalization
- ExtendedStrategyTuner: multi-dim Bayesian optimization (temperature, max_iterations, top_k, retrieval_mode)
- ExtendedStrategyConfig: expanded parameter space
- 20 tests passing
2026-06-06 22:42:54 +08:00
chiguyong d5998aaddd feat(evolution): U6 GEPA genetic algorithm evolution framework
- PromptChromosome: instructions + demos + constraints gene segments
- CrossoverOperator: paragraph-level text, demo, constraint crossover
- MutationOperator: LLM-driven instruction mutation + demo/constraint mutation
- GEPAPopulation: tournament selection, elite preservation, Pareto front
- FitnessScore: multi-objective (accuracy, latency, cost) with Pareto dominance
- 29 tests passing
2026-06-06 22:38:55 +08:00
chiguyong 1390bd8d6e feat(skills): U5 GEO Pipeline orchestration with DAG execution
- GEOPipeline: YAML-driven DAG pipeline with parallel/sequential execution
- PipelineStep with input_mapping ($.input.xxx, $.steps.name.output.xxx)
- Topological sort for execution groups, SharedWorkspace integration
- geo_full_pipeline.yaml: detect→analyze→optimize→track workflow
- 10 tests passing
2026-06-06 22:34:24 +08:00
chiguyong 23934602c0 feat(core): U4 multi-agent Orchestrator with SharedWorkspace
- Orchestrator: Orchestrator-Worker pattern with LLM-driven task decomposition
- SharedWorkspace: Redis-backed shared state with versioning and distributed locks
- SubTask dependency graph, parallel group building, result aggregation
- 16 tests passing
2026-06-06 22:25:12 +08:00
chiguyong 364fe6bd6d feat(memory): U3 EpisodicMemory ORM integration - EpisodeModel and session factory
- EpisodeModel ORM model with pgvector embedding support
- create_episodic_session_factory for async PostgreSQL sessions
- Server app.py now resolves session_factory from database_url config
- Graceful fallback when database_url not configured
2026-06-06 22:21:00 +08:00
chiguyong f16dcb5ebe feat(memory): U2 Contextual Retrieval - LLM-generated context prefixes for chunks
- ContextualChunker: generates context prefixes per chunk via LLM
- Integrated into HttpRAGService ingest with contextual_chunking option
- Caching, batch processing, graceful LLM failure handling
- 12 tests passing
2026-06-06 22:19:02 +08:00
chiguyong a6c9babfdc feat(memory): U1 RAG self-correction loop (CRAG)
- RelevanceScorer: keyword overlap + query coverage + retrieval score + length penalty
- RAGSelfCorrectionLoop: state machine driven retrieve-evaluate-correct-degrade cycle
- Integrated into MemoryRetriever with enable_self_correction option
- 21 tests passing
2026-06-06 22:16:23 +08:00
chiguyong 468dfd71e8 fix(test): adapt health check assertion to Phase 4 status value change 2026-06-06 21:56:30 +08:00
chiguyong 6e362a8ae7 feat(agentkit): Phase 4 enterprise production upgrade — 12 Implementation Units
Phase A (P0): EpisodicMemory pgvector search+EmbeddingCache, ReAct timeout+CancellationToken, evolution system fix (A/B test+LLMPromptOptimizer+StrategyTuner), AnthropicProvider native Messages API
Phase B (P1): RetryPolicy+CircuitBreaker, chat_stream fallback chain, WebSocket endpoint, SSE stream fix, Evolution+Memory API routes (7 endpoints), embedding cache+Enhanced Search per-KB degradation fix
Phase C (P2): GeminiProvider native generateContent API, Agent state lock+config hot-reload

Tests: 1301 passed, 18 skipped, 0 failed
2026-06-06 21:51:04 +08:00
chiguyong e33dc25ad3 feat(memory): RAG pipeline optimization — 5 Implementation Units
U1: QueryTransformer — LLM/rule-based query rewriting + sub-query decomposition
U2: HttpRAGService enhanced_search() — rerank + compression via /bases/{kb_id}/retrieve
U3: Structured context injection — source attribution headers in RAG results
U4: RetrieveKnowledgeTool — built-in tool for mid-reasoning knowledge retrieval
U5: Configurable retrieval params + per-KB weights + CJK token estimation

Config example:
  memory:
    retrieval:
      top_k: 5
      token_budget: 2000
      context_template: structured
    query_transform:
      enabled: true
      strategy: llm
    semantic:
      search_mode: enhanced
      use_rerank: true
      kb_weights:
        industry-kb-id: 1.2
        enterprise-kb-id: 0.8

Tests: 1037 passed, 18 skipped, 0 failed
2026-06-06 19:27:09 +08:00
chiguyong cd5b39087e feat(memory): add HttpRAGService for config-driven knowledge base integration 2026-06-06 18:36:05 +08:00
chiguyong 0456429beb fix(review): address all 14 P2 advisory findings 2026-06-06 18:20:46 +08:00
chiguyong 8620751864 fix(review): address P0+P1 findings from Tier 2 code review
P0: MemoryRetriever.retrieve score mutation fix
P1: Redis atomic Lua script, deprecated API fix, SQLite WAL mode,
Redis URL masking, UniqueConstraint, TraceRecorder completed flag,
EpisodicMemory recall improvement, LLMReflector sanitization,
A/B test safety, generator cleanup, ContextCompressor guards,
OpenAIEmbedder reuse, Pipeline failure handling, Metrics O(1),
Health check Redis PING, CLI skill loading, CORS config,
API key direct pass-through

Tests: 924 passed, 18 skipped, 0 failed
2026-06-06 17:57:47 +08:00
chiguyong f976fade99 docs: mark Phase 3 upgrade plan as completed 2026-06-06 17:18:07 +08:00
chiguyong f858d279f3 feat(agentkit): Phase 3 upgrade - persistence, memory, evolution, observability
10 Implementation Units across 3 phases:

Phase A - Infrastructure:
- U1: RedisTaskStore with Redis/memory backend + factory function
- U2: TraceRecorder for execution trace recording
- U3: PersistentEvolutionStore with SQLite backend

Phase B - Core Capabilities:
- U4: MemoryRetriever integration into ReAct engine
- U5: Embedder abstraction + EpisodicMemory vector search
- U6: LLMReflector for LLM-in-the-loop reflection
- U7: SkillPipeline for multi-skill orchestration

Phase C - Enhancement:
- U8: SKILL.md format + progressive disclosure levels
- U9: ContextCompressor + prompt cache rendering
- U10: Structured logging + metrics endpoint + enhanced health check

Tests: 924 passed, 18 skipped, 0 failed
2026-06-06 17:17:45 +08:00
chiguyong 74e2223153 feat(cli): pair command + doctor rename + client config priority
- health → doctor (better naming)
- agentkit pair --name <client> generates ak_live_ API key
- agentkit pair --list / --revoke for client management
- ClientConfig class: client config > init defaults > hardcoded
- README updated with pair usage + business system pairing guide
- 38 CLI tests passing
2026-06-06 13:08:14 +08:00
chiguyong 3cd6a73d86 feat(cli): Docker deployment + README CLI section
U5: docker-compose.yaml (agentkit + redis + postgres) + Dockerfile ENTRYPOINT
U6: README updated with CLI quick start + Docker deployment guide

31 CLI tests passing, no regression.
2026-06-06 12:47:47 +08:00
chiguyong b2709da08b feat(cli): AgentKit CLI with serve/version/health/task/skill/init/usage
U1: CLI framework (Typer) + serve/version/health commands + __main__.py + pyproject scripts
U2: task command group (submit/status/list/cancel) with remote mode
U3: skill command group (list/load/info) with local and remote modes
U4: init command (generates agentkit.yaml/.env.example/docker-compose/skills) + usage command

31 tests passing, TDD workflow.
2026-06-06 12:45:51 +08:00
chiguyong acec8ff743 feat(evolution): Phase A - lifecycle hooks + EvolutionConfig
U11: EvolutionMixin integrated into ConfigDrivenAgent lifecycle
  - on_task_complete triggers evolve_after_task
  - on_task_failed records failure patterns
  - Evolution errors never break main task flow

U12: EvolutionConfig added to SkillConfig
  - enabled, reflect_on_failure, auto_apply, min_quality_threshold
  - Backward compatible: defaults to enabled=False

21 new tests passing, no regression.
2026-06-06 12:05:56 +08:00