From c92baeca028f1ea8c315f796c203218ab5f3ade7 Mon Sep 17 00:00:00 2001 From: Vladimir Zagainov Date: Sat, 15 Aug 2026 01:22:50 +0300 Subject: [PATCH] add plan: full-implementation --- docs/plans/2026-08-15-full-implementation.md | 269 +++++++++++++++++++ 1 file changed, 269 insertions(+) create mode 100644 docs/plans/2026-08-15-full-implementation.md diff --git a/docs/plans/2026-08-15-full-implementation.md b/docs/plans/2026-08-15-full-implementation.md new file mode 100644 index 0000000..0dba3e9 --- /dev/null +++ b/docs/plans/2026-08-15-full-implementation.md @@ -0,0 +1,269 @@ +# Полная реализация согласно спецификации `docs/specification.md` + +## Overview +Implement the complete multimodal trip planning service as specified in `docs/specification.md`, covering all four development stages from MVP through polish. The implementation follows the lazy graph expansion architecture due to Yandex.Schedules API limitations (no full timetable dump). + +**Problem solved:** Users can find multimodal routes combining planes, trains, and buses with arbitrary transfer depth, automatic fallback to neighboring stations when main stations are closed, and map visualization — all within API quota constraints. + +**Key architectural decisions:** +- Lazy graph expansion with hub stations (instead of full RAPTOR, which would exhaust API quota) +- BFS/Dijkstra with depth limiting (4-5 transfers max) +- On-demand `/search` requests only for relevant station pairs +- Multi-layer TTL caching strategy +- Pareto-front ranking (time, transfers, cost) rather than single "optimal" route +- Station closure detection with automatic fallback + +## Context (from discovery) +- **Current state:** Lazy graph expansion partially implemented (commit edfc567): hub station selection, on-demand `/search`, transfer depth limiting, synthetic edge fallback, `ResetCircuitBreaker` helper +- **Files involved:** `internal/routing/graph.go`, `internal/routing/graph_test.go`, `internal/yandex/client.go`, `internal/yandex/client_test.go`, `internal/cache/`, `internal/storage/`, `cmd/api/`, `cmd/cron/` +- **Related patterns:** cache-aside, circuit breaker, transfer rules, MCT calculation, GeoJSON assembly +- **Dependencies:** PostgreSQL with PostGIS (optional), Redis with TTL, Yandex.Schedules API + +## Development Approach +- **Testing approach:** TDD (tests first) — all new code must have corresponding tests; tests are a required deliverable of every task, not optional +- All tests must pass before starting the next task — no exceptions +- Update plan file when scope changes during implementation +- Run tests after each change +- Maintain backward compatibility + +## Testing Strategy +- **Unit tests:** Required for every task — write tests for all new/modified functions, including success and error scenarios +- **Synthetic timetable fixtures:** Test routing algorithm on synthetic data without real API calls +- Mock external API calls in all tests +- Test cache-aside patterns thoroughly +- Validate MCT (Minimum Connection Time) calculations + +## Progress Tracking +- Mark completed items with `[x]` immediately when done +- Add newly discovered tasks with ➕ prefix +- Document issues/blockers with ⚠️ prefix +- Keep plan in sync with actual work done + +## What Goes Where +- **Implementation Steps** (`[ ]` checkboxes): tasks achievable within this codebase +- **Post-Completion** (no checkboxes): items requiring external action +- **Checkbox placement:** Checkboxes belong only in Task sections (`### Task N:`). Do not put checkboxes in Success criteria, Overview, or Context + +--- + +# Этап 1 — MVP (Minimum Viable Product) + +*Already partially implemented: lazy graph expansion, basic routing with single transport mode, basic caching* + +## Implementation Steps + +### Task 1: Refactor hub station selection [ ] +- [ ] Remove `Population` field from `HubStation` struct in `internal/routing/graph.go` +- [ ] Simplify `SelectHubStations` to use only `minOutgoingFlights` criterion +- [ ] Update all test criteria to match new hub selection logic (remove `minPopulation`) +- [ ] **Write tests:** TestSelectHubStations with various minOutgoingFlights values +- [ ] Run tests - must pass before task 2 + +### Task 2: Implement synthetic edge fallback in FindRoute [ ] +- [ ] Add synthetic edge fallback when lazy expansion fails in `FindRoute` method +- [ ] Create `addSyntheticEdgesForNode` function +- - [ ] Write tests: TestFindRouteWithSyntheticFallback +- [ ] Run tests - must pass before task 3 + +### Task 3: Add ResetCircuitBreaker helper [ ] +- [ ] Add `ResetCircuitBreaker` function to `internal/yandex/client.go` +- [ ] Update tests to use the new reset function +- [ ] **Write tests:** TestResetCircuitBreaker +- [ ] Run tests - must pass before task 4 + +### Task 4: Implement on-demand /search integration [ ] +- [ ] Integrate on-demand `/search` calls in lazy graph expansion +- [ ] Implement cache key generation and TTL policies +- [ ] Write tests: TestSearchRoutes_onDemand with circuit breaker reset +- [ ] Run tests - must pass before task 5 + +### Task 5: Transfer depth limiting [ ] +- [ ] Implement depth limiting in BFS/Dijkstra (max 4-5 transfers) +- [ ] Add transfer depth tracking in search options +- [ ] Write tests: TestFindRouteWithDepthLimiting +- [ ] Run tests - must pass before task 6 + +### Task 6: Pareto-front ranking [ ] +- [ ] Implement multi-criteria ranking (time, transfers, cost if available) +- [ ] Return set of non-dominated routes instead of single "optimal" +- [ ] Write tests: TestRouteParetoRanking +- [ ] Run tests - must pass before task 7 + +### Task 7: Basic caching layer [ ] +- [ ] Implement cache-aside pattern for `/search` results +- [ ] Add TTL policies: 2-6 hours for near-term dates, 7 days for far-term +- [ ] Write tests: TestCacheAsideSearch +- [ ] Run tests - must pass before task 8 + +### Task 8: Station status endpoint [ ] +- [ ] Implement `GET /v1/stations/{id}/status` endpoint +- [ ] Write tests: TestStationStatusEndpoint +- [ ] Run tests - must pass before task 9 + +**✅ Stage 1 Complete — MVP ready (basic single-mode routing with lazy expansion)** + +--- + +# Этап 2 — Мультимодальность и MCT (Minimum Connection Time) + +*Add planes and buses, synthetic edges with MCT rules, manual neighboring airports* + +## Implementation Steps + +### Task 9: Add bus transport type [ ] +- [ ] Add `TransportType` enum with values: `plane`, `train`, `bus` +- [ ] Update `Edge` struct to include `TransportType` +- [ ] Update routing algorithm to handle all three transport types +- [ ] **Write tests:** TestTransportTypesInGraph +- [ ] Run tests - must pass before task 10 + +### Task 10: Synthetic edges "город↔аэропорт" [ ] +- [ ] Implement synthetic edges for airport-city transfers +- [ ] Add constants for transfer time estimation (section 7.4) +- [ ] Mark synthetic edges in GeoJSON output (dashed line) +- [ ] **Write tests:** TestSyntheticAirportCityEdges +- [ ] Run tests - must pass before task 11 + +### Task 11: MCT rules implementation [ ] +- [ ] Create `transfer_rules` table migration +- [ ] Seed default MCT values (Section 7.4): + - airport_internal/through → 30 min + - airport_internal/separate → 60 min + - station_internal → 30 min + - airport_to_city/small → 60 min + - airport_to_city/million_plus → 90 min +- [ ] Implement `MinTransferTime` function reading from transfer rules +- [ ] Use MCT in routing algorithm for transfer validation +- [ ] **Write tests:** TestMCTCalculation, TestTransferRules +- [ ] Run tests - must pass before task 12 + +### Task 12: Manual neighboring stations [ ] +- [ ] Add `station_neighbors` table support +- [ ] Implement `internal/airports` package with geo + manual override +- [ ] Add `source` field (geo/manual) and `is_excluded` flag +- [ ] Update `cities/{id}/stations` endpoint to include neighbors when main station closed +- [ ] **Write tests:** TestNeighboringStations, TestStationNeighbors +- [ ] Run tests - must pass before task 13 + +### Task 13: Admin station status override [ ] +- [ ] Implement `POST /internal/admin/stations/{id}/status` endpoint +- [ ] Add authentication protection +- [ ] Allow manual status setting with `source: manual` +- [ ] Write tests: TestAdminStationStatus, TestAdminAuth +- [ ] Run tests - must pass before task 14 + +**✅ Stage 2 Complete — Multimodality + MCT operational** + +--- + +# Этап 3 — Глубокий поиск и автодетект (Deep Search + Closure Detection) + +*Lazy hub-based expansion to depth 4-5, Pareto ranking, auto-closure detection* + +## Implementation Steps + +### Task 14: Lazy hub expansion depth 4-5 [ ] +- [ ] Implement BFS/Dijkstra with explicit depth limiting +- [ ] Track transfer count at each step; stop when depth > 5 +- [ ] On expansion failure, add synthetic edges as fallback +- [ ] Write tests: TestLazyExpansionDepthLimit, TestFindRouteMaxTransfers +- [ ] Run tests - must pass before task 15 + +### Task 15: Pareto-front ranking integration [ ] +- [ ] Integrate multi-criteria ranking into route search results +- [ ] Sort by default "быстрее всего" (fastest) +- [ ] Add UI controls to switch to "меньше пересадок" / "дешевле" +- [ ] Write tests: TestParetoFrontGeneration +- [ ] Run tests - must pass before task 16 + +### Task 16: Auto station closure detection [ ] +- [ ] Implement daily cron job checking `/schedule` for monitored stations +- [ ] Track `zero_since` timestamp; if 0 flights for N=3 consecutive days → status `closed` +- [ ] Update `station_status` table with `zero_since`, `last_seen_flight` +- [ ] When station closed, automatically substitute neighboring stations +- [ ] Write tests: TestStationClosureDetection, TestAutoClosureChronology +- [ ] Run tests - must pass before task 17 + +### Task 17: Neighbor substitution in routing [ ] +- [ ] When station is closed, route automatically uses neighboring stations +- [ ] Update `GET /v1/cities/{id}/stations` to reflect closure status +- [ ] Write tests: TestRouteWithClosedStationSubstitution +- [ ] Run tests - must pass before task 18 + +### Task 18: GeoJSON route visualization [ ] +- [ ] Implement route-to-GeoJSON conversion +- [ ] Real segments: solid lines, color by transport type +- [ ] Synthetic segments: dashed lines +- [ ] Transfer point markers with popup info (connection time, type) +- [ ] Write tests: TestRouteGeoJSON, TestGeoJSONVisualization +- [ ] Run tests - must pass before task 19 + +**✅ Stage 3 Complete — Deep search + closure detection operational** + +--- + +# Этап 4 — Полировка (Polish) + +*Price consideration, flight change notifications, personalization* + +## Implementation Steps + +### Task 19: Price as routing criterion [ ] +- [ ] Investigate price source data from Yandex.Schedules +- [ ] If price data available, add as 4th routing criterion +- [ ] If not available, add marker "цена не указана" in UI +- [ ] Write tests: TestPriceInRouting (if applicable) +- [ ] Run tests - must pass before task 20 + +### Task 20: Flight change notifications [ ] +- [ ] Track already-built routes for status changes +- [ ] Implement re-search on significant changes (cancellation, major delay) +- [ ] Write tests: TestRouteReSearchOnChange +- [ ] Run tests - must pass before task 21 + +### Task 21: Personalization [ ] +- [ ] Add user preferences (saved cities, history of searches) +- [ ] Store preferences in Redis or Postgres +- [ ] Write tests: TestUserPreferences +- [ ] Run tests - must pass before task 22 + +### Task 22: Observability and metrics [ ] +- [ ] Add metrics: cache hit-rate per layer, API quota remaining, circuit breaker trips, average search time +- [ ] Add Prometheus metrics endpoints or logging structured +- [ ] Write tests: TestMetricsEndpoints +- [ ] Run tests - must pass before task 23 + +### Task 23: Full test suite and linter [ ] +- [ ] Run entire test suite: `go test ./...` +- [ ] Fix all linter issues: `go vet ./...` +- [ ] Verify test coverage meets standard (80%+) +- [ ] Fix any remaining issues +- [ ] **Final verification:** all checkboxes marked `[x]`, all tests passing + +**✅ Stage 4 Complete — Polish finished** + +--- + +# Post-Completion + +## Manual verification (if applicable) +- Manual UI/UX testing scenarios across all transport mode combinations +- Performance testing under load (simulate cold cache, warm cache scenarios) +- Security review considerations for admin endpoints + +## External system updates +- Consuming projects that may need updates after this library change +- Configuration changes in deployment systems (docker-compose, cron schedules) +- Third-party service integrations to verify (Yandex API access, Redis/PG connectivity) + +## Migration path from MVP to full +1. Stage 1 (MVP) → functional single-mode routing +2. Stage 2 → add planes/buses + MCT + manual neighbors +3. Stage 3 → lazy hub expansion + auto-closure + GeoJSON +4. Stage 4 → price, notifications, personalization, observability + +**Notes for ralphex:** +- Auto-move completed plan to `docs/plans/completed/` upon full task completion +- Each task MUST include tests as checklist items — they are not optional +- If tests cannot pass until a later task: write tests with TODO comment noting dependency, mark test checkbox as `[x] write tests ... (fails until Task X)`, do NOT skip test writing +- Update plan file when scope changes during implementation \ No newline at end of file