103 lines
6.0 KiB
Markdown
103 lines
6.0 KiB
Markdown
# Fix scanner engine wiring gap
|
||
|
||
## Overview
|
||
Fix the scanner engine wiring gap where the data producers (MusicBrainz/Navidrome sync) are implemented but never invoked by main.run(). Currently, main.run() only calls scanner.ScanAll and logs results, without calling musicbrainz.SyncArtistDiscography or any Navidrome sync, so the external_releases and local_albums tables are never populated by the running process.
|
||
|
||
This plan implements Approach A: Sequential sync then scan - modifying App.run() to call navidrome.SyncArtists, musicbrainz.SyncAll, then scanner.ScanAll in sequence.
|
||
|
||
## Context (from discovery)
|
||
- Files/components involved: cmd/naviwatcher/main.go, internal/musicbrainz/sync.go, internal/musicbrainz/syncall.go, internal/navidrome/sync.go, internal/scanner/scan.go
|
||
- Related patterns found: Existing sync functions are implemented but not wired in main execution flow
|
||
- Dependencies identified: MusicBrainz client, Navidrome client, database connections all already initialized in App
|
||
|
||
## Development Approach
|
||
- **Testing approach**: TDD (Tests first) - Write tests for the modified flow before implementing changes
|
||
- Complete each task fully before moving to the next
|
||
- Make small, focused changes
|
||
- **CRITICAL: every task MUST include new/updated tests** for code changes in that task
|
||
- Tests are not optional - they are a required part of the checklist
|
||
- Write unit tests for new functions/methods
|
||
- Write unit tests for modified functions/methods
|
||
- Add new test cases for new code paths
|
||
- Update existing test cases if behavior changes
|
||
- Tests cover both success and error scenarios
|
||
- **CRITICAL: all tests must pass before starting next task** - no exceptions
|
||
- **CRITICAL: update this plan file when scope changes during implementation**
|
||
- Run tests after each change
|
||
- Maintain backward compatibility
|
||
|
||
## Testing Strategy
|
||
- **Unit tests**: Required for every task (see Development Approach above)
|
||
- **E2E tests**: Project has existing test structure - maintain and extend as needed
|
||
- UI changes → add/update e2e tests in same task as UI code
|
||
- Backend changes supporting UI → add/update e2e tests in same task
|
||
- Treat e2e tests with same rigor as unit tests (must pass before next task)
|
||
- Store e2e tests alongside unit tests (or in designated e2e directory)
|
||
|
||
## Progress Tracking
|
||
- Mark completed items with `[x]` immediately when done
|
||
- Add newly discovered tasks with ➕ prefix
|
||
- Document issues/blockers with ⚠️ prefix
|
||
- Update plan if implementation deviates from original scope
|
||
- Keep plan in sync with actual work done
|
||
|
||
## What Goes Where
|
||
- **Implementation Steps** (`[ ]` checkboxes): tasks achievable within this codebase - code changes, tests, documentation updates
|
||
- **Post-Completion** (no checkboxes): items requiring external action - manual testing, changes in consuming projects, deployment configs, third-party verifications
|
||
- **Checkbox placement**: Belong ONLY in Task sections (`### Task N:` or `### Iteration N:`). Do NOT put checkboxes in Success criteria, Overview, or Context — they cause extra loop iterations.
|
||
|
||
## Implementation Steps
|
||
|
||
### Task 1: Understand current main.run() flow and identify integration points
|
||
- [x] Review current main.run() function in cmd/naviwatcher/main.go
|
||
- [x] Identify where navidrome.SyncArtists, musicbrainz.SyncAll, and scanner.ScanAll should be called
|
||
- [x] Examine existing App.syncAndScan() function to understand current scanning logic
|
||
- [x] Write tests to verify current behavior (scanner runs without data sync)
|
||
- [x] Run tests - must pass before proceeding
|
||
|
||
### Task 2: Modify App.run() to include data synchronization before scanning
|
||
- [x] Modify App.run() to call navidrome.SyncArtists(ctx, a.ndClient, a.db) first
|
||
- [x] Modify App.run() to call musicbrainz.SyncAll() with appropriate parameters
|
||
- [x] Modify App.run() to call scanner.ScanAll() after data synchronization
|
||
- [x] Ensure proper error handling and context propagation for each step
|
||
- [x] Write tests verifying the new synchronization sequence works correctly
|
||
- [x] Run tests - must pass before proceeding
|
||
|
||
### Task 3: Update App.syncAndScan() to use the new synchronized approach (optional refactor)
|
||
- [x] Evaluate whether App.syncAndScan() should be updated to use the new flow
|
||
- [x] If modifying, ensure it calls the same sync functions in the same order
|
||
- [x] Write tests to verify syncAndScan still works correctly
|
||
- [x] Run tests - must pass before proceeding
|
||
|
||
### Task 4: Verify end-to-end functionality works correctly
|
||
- [x] Create integration test that verifies data flows from sync -> scan -> notification
|
||
- [x] Test that artist settings (ignore_singles/ignore_compilations) are properly respected
|
||
- [x] Verify that external_releases and local_albums tables get populated
|
||
- [x] Run full test suite - must pass before proceeding
|
||
|
||
### Task 5: Update documentation to reflect the new data flow
|
||
- [x] Update CLAUDE.md if needed to document the new execution flow
|
||
- [x] Update any relevant comments in the code
|
||
- [x] Ensure documentation matches implementation
|
||
- [x] Run final validation
|
||
|
||
## Technical Details
|
||
- Data structures: Uses existing database.ExternalRelease, database.LocalAlbum types
|
||
- Parameters: Uses existing context.Context, database.DB, client instances
|
||
- Processing flow:
|
||
1. Sync artists from Navidrome (populate artist_settings)
|
||
2. Sync discographies from MusicBrainz (populate external_releases with filtering)
|
||
3. Sync albums from Navidrome (populate local_albums)
|
||
4. Scan for missing releases using fuzzy matching (produces MissingRelease results)
|
||
|
||
## Post-Completion
|
||
*Items requiring manual intervention or external systems - no checkboxes, informational only*
|
||
|
||
**Manual verification** (if applicable):
|
||
- Manual testing of the complete data pipeline with actual Navidrome and MusicBrainz services
|
||
- Performance testing under load to ensure synchronization doesn't block excessively
|
||
- Verification that configuration options still work as expected
|
||
|
||
**External system updates** (if applicable):
|
||
- Configuration documentation updates if new flags are added
|
||
- Deployment procedure updates if startup time characteristics change significantly |