fischer-agentkit

Commit Graph

Author	SHA1	Message	Date
chiguyong	11e2009cb8	feat(router): improve colloquial/mixed-lang routing, fix low-complexity IntentRouter bypass Key improvements: - Low-complexity queries (<0.3) now try IntentRouter keyword match before falling back to DIRECT_CHAT, fixing 0% F1 on keyword_match - SemanticRouter similarity_low lowered from 0.6 to 0.4 - Short text (<20 chars) uses effective_low = max(0.25, low - 0.15) - Short text with no semantic match forces LLM classify fallback - Added colloquial keywords to 7 skill YAMLs - Fixed code_reviewer.yaml output_schema placement - Fixed SemanticRouter build in e2e tests - Fixed base_url detection for bailian-coding API keys Results: keyword_match F1 0->60.87%, colloquial F1 0->100%, mixed_lang F1 0->100%	2026-06-15 23:54:57 +08:00
chiguyong	fa2a6dece2	feat(router): enable SemanticRouter + upgrade benchmark to L3/L5 - Enable SemanticRouter in agentkit.yaml (router.semantic.enabled: true) - Integrate SemanticRouter into e2e backtest (_build_real_components) - Add 8 new semantic test cases: 5 colloquial + 3 mixed-lang expressions - Add L3 output quality evaluation framework (LLM-as-Judge, 1-5 score) - Add L5 adaptive capability metrics (consistency rate from overfitting data) - Add OutputQualityObservation model and evaluate_output_quality() method - Report now includes L3 and L5 sections Results: 52 tests pass, description_match F1=66.67%, L5 adaptive rate=100%	2026-06-15 23:02:47 +08:00
chiguyong	e984b4c462	feat(router): optimize routing intelligence — ExecutionMode expansion, multi-candidate scoring, quality gate skill match - Expand ExecutionMode enum with REWOO/REFLEXION/PLAN_EXEC - Add _resolve_execution_mode() to respect skill.config.execution_mode - Rewrite IntentRouter._match_keywords() for multi-candidate scoring - Add QualityGate 5th dimension: skill_match validation with warning escalation - Calibrate HeuristicClassifier: low-complexity signals only when no high signals - Fix negation regex for Chinese text (avoid matching past punctuation) - Fix backtest mode_map normalization and .env loading - Add 61 unit tests (21 HeuristicClassifier + 14 IntentRouter + 13 QualityGate + 13 existing) Results: execution_mode_accuracy 9.09%→36.36%, skill_routing_F1 66.67%→77.78%	2026-06-15 22:43:13 +08:00

Author

SHA1

Message

Date

chiguyong

11e2009cb8

feat(router): improve colloquial/mixed-lang routing, fix low-complexity IntentRouter bypass

Key improvements:
- Low-complexity queries (<0.3) now try IntentRouter keyword match
  before falling back to DIRECT_CHAT, fixing 0% F1 on keyword_match
- SemanticRouter similarity_low lowered from 0.6 to 0.4
- Short text (<20 chars) uses effective_low = max(0.25, low - 0.15)
- Short text with no semantic match forces LLM classify fallback
- Added colloquial keywords to 7 skill YAMLs
- Fixed code_reviewer.yaml output_schema placement
- Fixed SemanticRouter build in e2e tests
- Fixed base_url detection for bailian-coding API keys

Results: keyword_match F1 0->60.87%, colloquial F1 0->100%, mixed_lang F1 0->100%

2026-06-15 23:54:57 +08:00

chiguyong

fa2a6dece2

feat(router): enable SemanticRouter + upgrade benchmark to L3/L5

- Enable SemanticRouter in agentkit.yaml (router.semantic.enabled: true)
- Integrate SemanticRouter into e2e backtest (_build_real_components)
- Add 8 new semantic test cases: 5 colloquial + 3 mixed-lang expressions
- Add L3 output quality evaluation framework (LLM-as-Judge, 1-5 score)
- Add L5 adaptive capability metrics (consistency rate from overfitting data)
- Add OutputQualityObservation model and evaluate_output_quality() method
- Report now includes L3 and L5 sections

Results: 52 tests pass, description_match F1=66.67%, L5 adaptive rate=100%

2026-06-15 23:02:47 +08:00

chiguyong

e984b4c462

feat(router): optimize routing intelligence — ExecutionMode expansion, multi-candidate scoring, quality gate skill match

- Expand ExecutionMode enum with REWOO/REFLEXION/PLAN_EXEC
- Add _resolve_execution_mode() to respect skill.config.execution_mode
- Rewrite IntentRouter._match_keywords() for multi-candidate scoring
- Add QualityGate 5th dimension: skill_match validation with warning escalation
- Calibrate HeuristicClassifier: low-complexity signals only when no high signals
- Fix negation regex for Chinese text (avoid matching past punctuation)
- Fix backtest mode_map normalization and .env loading
- Add 61 unit tests (21 HeuristicClassifier + 14 IntentRouter + 13 QualityGate + 13 existing)

Results: execution_mode_accuracy 9.09%→36.36%, skill_routing_F1 66.67%→77.78%

2026-06-15 22:43:13 +08:00

3 Commits