Commit Graph

65 Commits

Author SHA1 Message Date
chiguyong 79139bc504 chore: geo production readiness improvements 2026-06-04 22:08:06 +08:00
chiguyong 435fec2b00 chore: update production env example with fischerai.cn domains
- NEXT_PUBLIC_API_URL → https://geo.api.fischerai.cn
- CORS_ORIGINS → https://geo.fischerai.cn
- Add BACKEND_INTERNAL_URL for server-side API calls
2026-06-04 16:37:42 +08:00
chiguyong 66e4bff60d chore: add Docker log rotation to all production services
- Add json-file log driver with rotation to all 4 services
- Backend/frontend: max-size 50m, max-file 5
- PostgreSQL: max-size 30m, max-file 3
- Redis: max-size 20m, max-file 3
- Prevents disk exhaustion from unbounded container logs
2026-06-04 14:22:33 +08:00
chiguyong 6b90fb5cd6 refactor: unify Redis connection pool across all modules
- Create app/core/redis.py with global get_redis() singleton
- Replace 4 independent Redis connections:
  - cache.py: use get_redis() instead of own aioredis.from_url
  - dispatcher.py: use get_redis() instead of own connection
  - health_checker.py: use get_redis() instead of per-check connection
  - rate_limit.py: RedisRateLimitBackend uses get_redis() instead of own connection
- Replace main.py readiness endpoint to use get_redis()
- Add close_redis() in FastAPI lifespan shutdown
- Remove unused aioredis imports from health_checker.py and main.py
2026-06-04 14:21:14 +08:00
chiguyong bdf351977b chore: complete test file migration (delete old, add new paths) 2026-06-04 14:07:27 +08:00
chiguyong c253ccd794 chore: add coverage/test artifacts to .gitignore, add plan docs
- Add .coverage, htmlcov/, test-results/, playwright-report/ to .gitignore
- Add Plan 008 (production readiness) and Plan 009 (production hardening + test infra)
2026-06-04 14:06:44 +08:00
chiguyong d14d500e02 feat: E2E shared fixtures, API mock layer, interaction + error tests
- Create shared Playwright fixtures (authenticatedPage, mockApi, etc.)
- Create API mock layer using page.route() interception
- Refactor 14 existing test files to use shared fixtures
- Add 5 new E2E test files:
  - health-score-interaction: score dimension details, tab switching
  - citation-flow: search, filter, detail view, export
  - competitor-interaction: add/compare/remove competitors
  - error-states: API 500, offline, empty data, 401 redirect
  - knowledge-interaction: create KB, upload/delete documents
2026-06-04 14:06:25 +08:00
chiguyong 2a46f89a8a feat: tag validation logic + pytest-cov auto coverage
- Implement min_tags/max_tags validation in platform_rules.py
- Support tags as list or comma-separated string
- Add tags field to ContentValidateRequest schemas
- Pass tags through API and service layers
- Configure pytest-cov in pyproject.toml (auto coverage on pytest run)
2026-06-04 14:06:00 +08:00
chiguyong 4507afbbfd fix: CI workflow concurrency groups and step ID references
- Add concurrency groups to ci.yml and pr-check.yml to cancel redundant runs
- Fix pr-check.yml step IDs to kebab-case (invalid with spaces)
- Rename type-check step to style-check (ruff)
2026-06-04 14:05:17 +08:00
chiguyong c428728742 feat: frontend Sentry integration + auth error handling
- Add sentry.client.config.ts and sentry.server.config.ts
- Add @sentry/nextjs to package.json
- Replace TODO:SENTRY in ErrorBoundary with actual Sentry.captureException
- Add console.error + Sentry reporting in auth.ts authorize and refreshAccessToken
- Enable TypeScript strict checks in production builds only
2026-06-04 14:04:55 +08:00
chiguyong 3737a90471 feat: Sentry integration + rate limiter dual backend
- Initialize Sentry SDK in FastAPI (auto-disabled when DSN empty)
- Add sentry_sdk.set_measurement in metrics middleware
- Add sentry-sdk[fastapi] to requirements
- Refactor rate_limit.py: abstract RateLimitBackend + MemoryBackend + RedisBackend
- Redis backend uses sorted set sliding window with pipeline atomicity
- Memory backend adds asyncio cleanup task to prevent memory growth
- Auto-fallback to memory when Redis unavailable
- Add RATE_LIMIT_BACKEND config (default: memory)
2026-06-04 14:04:36 +08:00
chiguyong ee8578c3d7 chore: split Dockerfile into runtime+e2e stages, add healthchecks
- Split Dockerfile: runtime stage (no Playwright) + e2e stage (adds Chromium)
- Add healthcheck to backend and frontend in docker-compose.prod.yml
- Fix REDIS_URL default to empty string (no hardcoded password)
- Add RATE_LIMIT_BACKEND=redis to production env example
- Remove deprecated version field from docker-compose.prod.yml
2026-06-04 14:04:11 +08:00
chiguyong 331f6bce80 fix: add missing Agent exports in __init__.py
Export CompetitorAnalyzerAgent, MonitorAgent, SchemaAdvisorAgent
that were implemented but missing from the module's public API.
2026-06-04 14:03:47 +08:00
chiguyong fb4778541c fix: improve playwright config for Docker environment stability 2026-06-02 23:54:12 +08:00
chiguyong bfc67fce09 fix: add requests dependency for payment SDK integration 2026-06-02 22:24:21 +08:00
chiguyong 811ddcdbd7 chore: mark launch sprint plan as completed 2026-06-02 21:51:17 +08:00
chiguyong 3711f1641a feat: payment SDK integration, attribution window config, E2E tests, CI integration, rules center tests
- WeChat Pay V3: real API calls with HMAC-SHA256 signing, AES-GCM callback decryption
- Alipay: real API calls with RSA2 signing, WAP payment support
- Both gateways fallback to MockGateway when unconfigured
- Attribution window configurable via ATTRIBUTION_WINDOW_DAYS env var (default 28)
- 30 platform-specific rule test cases (WeChat/Zhihu/Xiaohongshu/Baijiahao/Toutiao/Douyin)
- Fixed clickbait detection bug (character-level to word-level matching)
- E2E tests for diagnosis-strategy and content-monitoring flows
- CI: e2e-test job, bandit security scan, npm audit, performance baseline
2026-06-02 21:50:26 +08:00
chiguyong 680d8fc9e1 chore: unify test dirs, remove debug scripts, add content/diagnosis/attribution fixtures 2026-06-02 21:15:15 +08:00
chiguyong 98d6528dc2 chore: switch prod PostgreSQL to pgvector image and secure .env.production 2026-06-02 21:09:13 +08:00
chiguyong da8d1e7667 chore: add launch sprint brainstorm requirements and implementation plan 2026-06-02 21:06:06 +08:00
chiguyong feb2bb2af1 merge: Plan 006 — GEO frontend visualization (U1-U7) 2026-06-02 08:14:24 +08:00
chiguyong 5c7d921825 docs: mark plan 006 as completed 2026-06-02 08:12:14 +08:00
chiguyong f182e166dc feat: U7 — citation export, agent config panel, trends insight page, schema suggestion page 2026-06-02 08:11:43 +08:00
chiguyong 01e83b3589 feat: U3-U6 — onboarding auto-create monitoring, citation stats viz, health score page, detection tasks + dashboard agent activity 2026-06-02 07:59:08 +08:00
chiguyong 37ecd39a60 feat: monitoring page refactor + competitor analysis page (U1, U2)
- Refactor monitoring page: dual-tab (records + alerts), connect monitoringApi
- Replace sidebar '数据监测' nav with '品牌监测' → /dashboard/monitoring
- Add '竞品分析' nav item → /dashboard/competitors
- Create competitor API module with 8 endpoints
- Create competitor analysis page with radar chart, gap scores, recommendations
2026-06-02 07:41:04 +08:00
chiguyong 45e151fc31 fix: frontend quality improvements
- Fix ONBOARDING_STEPS count (5→6) to match actual flow
- Unify OnboardingState type (remove duplicate from page.tsx)
- Replace raw fetch with fetchWithAuth in health-score.ts
- Extract shared utils (round, getStatusColor, DIMENSION_ICONS) to lib/utils/health-score.ts
- Fix Step5 handleComplete silent failure on error
- Remove console.error from Step2/Step4/Step5
- Remove unused props from Step3Platforms
- Fix TS errors in agents/page.tsx and strategy/page.tsx
- Exclude test files from tsc (handled by vitest)
2026-06-01 23:41:27 +08:00
chiguyong 218ece564d fix: Docker deployment optimization
- Replace postgres:15-alpine with pgvector/pgvector:pg15 (built-in vector extension)
- Remove init-db.sh (pgvector image includes extension, SQL init script instead)
- Add multi-stage Dockerfile for backend (development/runtime targets)
- Add development stage to frontend Dockerfile
- Update .env.example with correct passwords and Docker-internal URLs
- Add POSTGRES_PASSWORD/REDIS_PASSWORD to .env for Docker Compose
- Use named volume for frontend node_modules
- Add backend healthcheck to docker-compose.yml
2026-06-01 23:10:29 +08:00
chiguyong 33aecc0cb1 fix: squash Alembic migrations into single complete initial migration (48 tables, 99 TIMESTAMPTZ columns) 2026-06-01 22:07:01 +08:00
chiguyong ed400e63d8 docs: mark Plan 004 and Plan 005 as completed 2026-06-01 21:47:05 +08:00
chiguyong 9fb4dad215 Merge branch 'chore/geo-tech-debt-cleanup' — Plan 005: tech debt cleanup sprint 2026-06-01 21:44:44 +08:00
chiguyong 47719b25ce feat: deployment security hardening
- Redis: add password authentication (requirepass), update healthcheck
- PostgreSQL: replace hardcoded password with env variable
- Docker Compose: remove obsolete version field, use env vars for credentials
- Add .env.production.example template with strong password placeholders
- Update all .env files with new credentials and ports
2026-06-01 21:34:13 +08:00
chiguyong d5f752f419 fix: add DateTime(timezone=True) to analytics.py (4 columns), update migration and port config
- analytics.py: 4 DateTime columns now timezone-aware (99 total, 0 remaining)
- Migration script updated with publish_records, content_metrics, optimization_insights
- Docker Compose: db port 5433, redis port 6380 (avoid conflicts with fischerx)
- .env files: DATABASE_URL and REDIS_URL updated to new ports
- alembic.ini: updated to localhost:5433
2026-06-01 21:26:39 +08:00
chiguyong eabd23d093 feat: add Alembic migration for timezone-aware datetime columns (95 columns, 38 tables)
- Convert TIMESTAMP WITHOUT TIME ZONE to TIMESTAMP WITH TIME ZONE
- Uses AT TIME ZONE 'UTC' for safe data conversion
- Covers all models: users, brands, queries, citations, agents, etc.
- Includes downgrade path back to TIMESTAMP WITHOUT TIME ZONE
2026-06-01 21:08:56 +08:00
chiguyong f1a8b69c2a fix: unify frontend API client - add blob support to fetchWithAuth, eliminate raw fetch calls
- Extend fetchWithAuth with responseType parameter ('json' | 'blob')
- reports.ts: PDF/CSV export now uses fetchWithAuth blob mode
- reports/page.tsx: remove duplicate API_BASE, use fetchWithAuth for CSV export
- lifecycle/new/page.tsx: replace raw fetch with fetchWithAuth for quick-start POST
2026-06-01 20:54:12 +08:00
chiguyong 792d9ebe53 fix: add DateTime(timezone=True) to Batch 1c auxiliary path models (9 files, 23 columns) and remove orphan monitoring_record.py 2026-06-01 20:48:39 +08:00
chiguyong 14cafa66c6 fix: add DateTime(timezone=True) to Batch 1b agent framework path models (7 files, 28 columns) 2026-06-01 20:41:31 +08:00
chiguyong 289f04302f fix: add DateTime(timezone=True) to Batch 1a core monetization path models (11 files, 32 columns) 2026-06-01 20:39:14 +08:00
chiguyong 4168aca107 chore: Plan 005 - tech debt cleanup sprint (timezone, API client, security) 2026-06-01 20:36:10 +08:00
chiguyong 4f86f2bd62 chore: Plan 004 - launch readiness sprint (timezone fixes, health check, JWT secret) 2026-06-01 20:35:56 +08:00
chiguyong 3bd848ee36 feat: 添加 pgvector 初始化脚本和数据库 schema 初始化工具
- backend/init-db.sh: Docker 首次启动时编译安装 pgvector v0.5.1
- backend/init_schema.py: create_all + stamp head 初始化数据库
- docker-compose.yml: 挂载 init-db.sh 到 /docker-entrypoint-initdb.d/
2026-06-01 14:37:02 +08:00
chiguyong 394ddfbc61 Merge branch 'feat/geo-monetization-closed-loop' into main 2026-06-01 10:07:07 +08:00
chiguyong d501262119 chore: plan-003 收尾 — FK类型修复、User模型补全、迁移脚本清理、前端构建修复
- fix(backend): 修复 FK 类型不匹配
  - TrendInsight.brand_id: String → Uuid (匹配 brands.id)
  - AgentConfig.updated_by: Uuid → String(36) (匹配 users.id)
  - KnowledgeBase.created_by: Uuid → String(36) (匹配 users.id)
  - MonitoringRecord.user_id: Uuid → String(36) (匹配 users.id)

- fix(backend): User 模型添加 plan/max_queries 列定义
  - register_user() 设置默认 plan=free, max_queries=5

- chore(backend): 清理 Alembic 迁移脚本
  - 只保留 diagnosis_records/attribution_records/payment_orders 3表变更

- fix(frontend): 创建缺失 UI 组件 (textarea, progress, use-toast)
- fix(frontend): ESLint 规则降级为 warn (预存问题不阻塞构建)
- chore: 更新计划003状态 active → completed
2026-06-01 09:50:52 +08:00
chiguyong b41da42d74 feat(geo): U1-U9 monetization closed loop implementation
U1: GEO diagnosis auto data collection (DataCollectorService + 11 tests)
U2: Free GEO health score public page (HealthScoreAPI + 7 tests)
U3: Onboarding redesign with conversion layer (Step0 + UpgradePrompt + 14 tests)
U4: Real payment integration (WeChat/Alipay gateway + PaymentOrder + 12 tests)
U5: AI content generation & distribution (Publishers + PublishEngine + 11 tests)
U6: Attribution system & ROI reports (AttributionEngine + ROICalculator + 12 tests)
U7: Email integration & Dashboard monetization UI (EmailScheduler + templates + 22 tests)
U8: Integration tests & shared fixtures (monetization flow + fixture consolidation + 12 tests)
U9: E2E smoke tests (HealthScorePage + acquisition/core flow + 4 E2E cases)

Total: 101 new backend tests passing, 4 E2E test cases
All third-party integrations use mock mode by default with config center management
2026-06-01 07:39:05 +08:00
chiguyong 900a90ba84 fix: 修复数据库schema兼容性和E2E测试
后端修复:
- User模型添加organization_id和role字段,匹配Prisma数据库schema
- SQLAlchemy模型FK类型从UUID改为String(36),匹配users.id的text类型
- lifespan中create_all改为SELECT 1,避免与Prisma schema冲突
- 数据库驱动从asyncpg切换到psycopg,修复macOS Unix socket问题
- auth API使用UserResponse.from_user()处理属性映射
- 修复auth service使用正确的列名(password/username)

E2E测试修复:
- hasProjects()先等待页面加载完成再检测空/错误状态
- loginAndWait增加60s超时和重试逻辑,解决NextAuth间歇性超时
- login-redirect-system-chrome添加browserName skip和重试
- login-redirect子页面测试使用domcontentloaded等待策略
- Dashboard空状态下依赖项目的测试正确skip
- playwright.config.ts reuseExistingServer硬编码为true
2026-05-27 20:57:49 +08:00
chiguyong 0a39ce6ef1 fix: 审计问题修复第二轮
安全:
- H2: CORS生产环境localhost警告验证器
- M12: JWT_SECRET已有≥32字符验证(确认)

代码质量:
- H4: 11处Any类型替换为具体联合类型
- H5: 4个模型测试文件(47个测试),模型覆盖率32%→64%
- M11: Alembic迁移脚本(6个缺失表),修复迁移链分支

测试: 717 passed
2026-05-26 07:34:07 +08:00
chiguyong aeaa50e89e fix: 审计发现的问题修复
API一致性修复:
- C1: 新增organization.py路由(/api/v1/organization/*)
- C2: 修复suggestions API路径(/api/v1/brands/*而非/api/v1/suggestions/*)
- H7: 修复platforms路由双重前缀(/api/v1/platforms而非/api/v1/api/platforms)

密钥管理改进:
- H3: APIKeyManager支持双密钥(dict格式),文心一言适配器使用KeyManager
- H8: 新增APIKeyFilter日志过滤器,拦截key=和Bearer token

异常处理改进:
- H1: batch_query.py改为httpx.HTTPError分层处理
- H1: database.py改为SQLAlchemyError并抛出ConnectionError
- H1: lifecycle.py和usage_tracker添加日志记录

测试: 764 passed
2026-05-25 23:33:25 +08:00
chiguyong fe4ba39514 fix: 消除所有Mock/Stub/假数据,确保业务流程使用真实数据
M1-引用检测核心:
- 删除llm_adapter._get_mock_result()方法
- ENABLE_LLM=False时抛出LLMAdapterError而非返回随机数据
- ENABLE_LLM默认值改为True
- 修复旧测试适配新行为

M2-知识库RAG:
- knowledge.py不再默认使用MockEmbedder
- 动态从APIKeyManager获取OpenAI Key
- 无Key时返回503+明确错误信息
- 有Key时使用OpenAIEmbedder

M3-AI引擎页面:
- 删除MOCK_AI_ENGINES_RESPONSE fallback
- 查询失败时显示错误状态

M4-组织管理页面:
- 删除MOCK_ORG_INFO和MOCK_MEMBERS
- API返回空时显示空状态

M5-首页Agent卡片:
- 删除MOCK_AGENTS硬编码
- 替换为功能开发中占位

M6-平台规则历史:
- 实现PlatformRuleVersion模型
- 实现版本对比API (diff)
- 实现历史记录查询API (history)
- 删除2个TODO注释

M7-知识图谱批量构建:
- 实现批量创建实体API
- 空输入验证+批量大小限制
- 删除TODO注释
- 修复路由双重前缀问题

测试: 643 passed (核心)
2026-05-25 21:51:48 +08:00
chiguyong 4cc8f73bb4 feat: API Key管理+用量追踪完整功能链路v2(真实可用)
持久化存储:
- APIKey模型 + APIKeyRepository(SQLAlchemy)
- UsageRecord模型 + UsageRepository(SQLAlchemy)

API Key验证:
- KeyVerifier服务(真正调用引擎API验证)
- 支持9个引擎的真实性验证

加密存储:
- KeyEncryption服务(Fernet AES加密)
- 环境变量API_KEY_ENCRYPTION_KEY

用量追踪:
- UsageRecorder自动记录查询用量
- 按引擎/按日聚合(修复by_day空dict)
- UserQuotaService支持套餐配额(free:10/basic:50/pro:200/enterprise:1000)

集成修复:
- AI引擎适配器使用APIKeyManager获取Key(用户Key>系统Key>环境变量)
- SmartRouter与APIKeyManager集成(过滤无Key引擎)
- BatchQueryService自动记录用量并传递用户上下文
- 所有适配器支持引擎特定代理环境变量

前端:
- usage页面替换MOCK为真实API调用
- 显示加载/错误/空状态

测试: 630 passed
2026-05-25 20:43:08 +08:00
chiguyong 290ef5a273 feat: API Key管理+用量追踪完整功能链路
后端:
- api_key_manager: 加密存储、脱敏显示、优先级排序、降级策略、Key可用性检测
- smart_router: 分层路由策略(FREE→LOW_COST→MID_COST→HIGH_COST)、国内引擎优先
- usage_tracker: Token消耗统计、成本计算、配额预警(ok/warning/exceeded)
- API端点: /api/v1/api-keys/ (CRUD+verify), /api/v1/usage/ (summary+quota+by-engine)
- 测试: 19个API测试 + 37个服务测试全部通过

前端:
- settings页面API配置标签页: 按国内外分组展示9个引擎、添加/删除/验证Key
- usage页面: 配额概览(环形进度)、用量趋势图(Recharts)、引擎分布饼图、明细表格
- 修复API路径与后端不一致问题
- 修复alerts API参数顺序问题
2026-05-25 15:37:33 +08:00
chiguyong 41c2994222 feat: API Key管理+智能路由+用量追踪 - 性价比最优方案
后端(TDD):
- API Key管理服务(加密存储+脱敏显示+优先级+降级策略)
  - 用户Key > 系统Key > 环境变量Key
  - Key可用性检测
  - Key过期处理
- 智能路由服务(分层路由+成本优先级)
  - FREE层: DeepSeek/通义千问/文心一言
  - LOW_COST层: Kimi/豆包/Gemini
  - MID_COST层: 腾讯元宝
  - HIGH_COST层: ChatGPT/Perplexity(用户自备Key)
  - 国内引擎优先
  - 成本估算
  - 推荐引擎组合
- 用量追踪服务(记录+统计+配额预警)
  - 日/周/月汇总
  - 按引擎/品牌统计
  - 成本计算
  - 配额预警(ok/warning/exceeded)
- 36+37=73个测试全部通过
2026-05-25 14:52:31 +08:00