Compare commits
7 Commits
musicbrain
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
| 8bba1bf22f | |||
| 7b88a6af17 | |||
| f01b5b2b36 | |||
| c434a842fb | |||
| cc04cad6e0 | |||
| e5631ce009 | |||
| 9d24730efa |
@@ -1,77 +1,99 @@
|
|||||||
name: Build and Push Docker Image
|
name: CI/CD
|
||||||
|
|
||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
branches: [ "**" ]
|
branches: [ main, master ]
|
||||||
tags: [ 'v*' ]
|
tags: [ 'v*' ]
|
||||||
pull_request:
|
pull_request:
|
||||||
branches: [ master ]
|
branches: [ main, master ]
|
||||||
|
|
||||||
env:
|
env:
|
||||||
# Docker image configuration - using your Gitea registry
|
GO_VERSION: "1.25"
|
||||||
REGISTRY: gitea.mrixs.me
|
REGISTRY: gitea.mrixs.me
|
||||||
IMAGE_NAME: naviwatcher
|
IMAGE_NAME: naviwatcher
|
||||||
|
# Using a fixed username since GITEA_REPOSITORY_OWNER may not be available
|
||||||
|
# Replace 'Mrixs' with your actual Gitea username
|
||||||
|
USERNAME: mrixs
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
|
test:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- uses: actions/setup-go@v5
|
||||||
|
with:
|
||||||
|
go-version: ${{ env.GO_VERSION }}
|
||||||
|
|
||||||
|
- name: Cache Go modules
|
||||||
|
uses: actions/cache@v4
|
||||||
|
with:
|
||||||
|
path: |
|
||||||
|
~/go/pkg/mod
|
||||||
|
~/.cache/go-build
|
||||||
|
key: go-test-${{ runner.os }}-${{ hashFiles('go.sum') }}
|
||||||
|
restore-keys: |
|
||||||
|
go-test-${{ runner.os }}-
|
||||||
|
|
||||||
|
- name: Download Go modules
|
||||||
|
run: go mod download
|
||||||
|
|
||||||
|
- name: Run tests
|
||||||
|
run: go test ./... -v -race -cover
|
||||||
|
|
||||||
build:
|
build:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
permissions:
|
needs: test
|
||||||
contents: read
|
|
||||||
packages: write # Needed for writing to GitHub Packages registry
|
|
||||||
|
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout repository
|
- uses: actions/checkout@v4
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Go
|
- uses: actions/setup-go@v5
|
||||||
uses: actions/setup-go@v4
|
|
||||||
with:
|
with:
|
||||||
go-version: '1.25'
|
go-version: ${{ env.GO_VERSION }}
|
||||||
cache: true
|
|
||||||
|
|
||||||
- name: Verify dependencies
|
- name: Cache Go modules and build cache
|
||||||
run: |
|
uses: actions/cache@v4
|
||||||
go mod tidy
|
|
||||||
go mod verify
|
|
||||||
|
|
||||||
- name: Run unit tests
|
|
||||||
run: go test ./... -v -coverprofile=coverage.out
|
|
||||||
|
|
||||||
- name: Upload coverage report
|
|
||||||
if: always()
|
|
||||||
uses: actions/upload-artifact@v4
|
|
||||||
with:
|
with:
|
||||||
name: coverage-report
|
path: |
|
||||||
path: coverage.out
|
~/go/pkg/mod
|
||||||
|
~/.cache/go-build
|
||||||
|
key: go-build-${{ runner.os }}-${{ hashFiles('go.sum') }}
|
||||||
|
restore-keys: |
|
||||||
|
go-build-${{ runner.os }}-
|
||||||
|
|
||||||
|
- name: Download Go modules
|
||||||
|
run: go mod download
|
||||||
|
|
||||||
|
- name: Build binary
|
||||||
|
run: go build -o naviwatcher ./cmd/naviwatcher
|
||||||
|
|
||||||
|
docker:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
needs: build
|
||||||
|
if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/v')
|
||||||
|
permissions:
|
||||||
|
packages: write
|
||||||
|
contents: read
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
- name: Set up Docker Buildx
|
- name: Set up Docker Buildx
|
||||||
uses: docker/setup-buildx-action@v3
|
uses: docker/setup-buildx-action@v3
|
||||||
|
|
||||||
- name: Login to Gitea Container Registry
|
- name: Login to Gitea Registry
|
||||||
uses: docker/login-action@v3
|
uses: docker/login-action@v3
|
||||||
with:
|
with:
|
||||||
registry: ${{ env.REGISTRY }}
|
registry: ${{ env.REGISTRY }}
|
||||||
username: ${{ gitea.repository_owner }}
|
username: ${{ env.USERNAME }}
|
||||||
password: ${{ secrets.PACKAGES_TOKEN }}
|
password: ${{ secrets.PACKAGES_TOKEN }}
|
||||||
|
|
||||||
- name: Extract Docker metadata
|
- name: Build and push
|
||||||
id: meta
|
|
||||||
uses: docker/metadata-action@v5
|
|
||||||
with:
|
|
||||||
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
|
||||||
tags: |
|
|
||||||
type=ref,event=branch
|
|
||||||
type=ref,event=pr
|
|
||||||
type=semver,pattern={{version}}
|
|
||||||
type=semver,pattern={{major}}.{{minor}}
|
|
||||||
type=sha
|
|
||||||
|
|
||||||
- name: Build and push Docker image
|
|
||||||
uses: docker/build-push-action@v5
|
uses: docker/build-push-action@v5
|
||||||
with:
|
with:
|
||||||
context: .
|
context: .
|
||||||
push: true
|
push: true
|
||||||
tags: ${{ steps.meta.outputs.tags }}
|
tags: |
|
||||||
labels: ${{ steps.meta.outputs.labels }}
|
${{ env.REGISTRY }}/${{ env.USERNAME }}/${{ env.IMAGE_NAME }}:latest
|
||||||
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-buildcache
|
${{ env.REGISTRY }}/${{ env.USERNAME }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
|
||||||
cache-to: type=inline,mode=max
|
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.USERNAME }}/${{ env.IMAGE_NAME }}:buildcache
|
||||||
|
cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.USERNAME }}/${{ env.IMAGE_NAME }}:buildcache,mode=max
|
||||||
|
|||||||
74
docs/FILTER_FLOW.md
Normal file
74
docs/FILTER_FLOW.md
Normal file
@@ -0,0 +1,74 @@
|
|||||||
|
# Filter Flow Documentation
|
||||||
|
|
||||||
|
This document explains how the `ignore_singles` and `ignore_compilations` settings propagate through the NaviWatcher system.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The application implements a dual-path filtering approach to balance storage efficiency with real-time responsiveness to user preference changes:
|
||||||
|
|
||||||
|
1. **MusicBrainz Sync Path (Store-time filtering)**:
|
||||||
|
- Applies filtering when caching release groups from the MusicBrainz API
|
||||||
|
- Stores only filtered results in the `external_releases` table
|
||||||
|
- Reduces database size by excluding unwanted release types upfront
|
||||||
|
|
||||||
|
2. **Scanner Path (Read-time filtering)**:
|
||||||
|
- Applies filtering when retrieving cached data for comparison
|
||||||
|
- Ensures changes to `ignore_singles`/`ignore_compilations` take effect immediately
|
||||||
|
- Provides real-time responsiveness without waiting for cache expiry
|
||||||
|
|
||||||
|
## Data Flow
|
||||||
|
|
||||||
|
1. **Artist Settings Storage**:
|
||||||
|
- Navidrome artist sync populates the `artist_settings` table
|
||||||
|
- Stores `ignore_singles` and `ignore_compilations` boolean flags per artist
|
||||||
|
|
||||||
|
2. **MusicBrainz Synchronization**:
|
||||||
|
- `SyncArtistDiscography` retrieves artist settings via `getArtistFilterOptions`
|
||||||
|
- Applies filtering using `ApplyTypeToggles` before storing in `external_releases`
|
||||||
|
- On cache hits, re-applies filter to ensure freshness of user preferences
|
||||||
|
|
||||||
|
3. **Scanning Process**:
|
||||||
|
- `ScanArtist settings via `GetArtistSettings`
|
||||||
|
- Creates `TypeFilter` with current `ignore_singles`/`ignore_compilations` values
|
||||||
|
- Applies filter during `FindMissingReleases` via `filter.suppressed()` check
|
||||||
|
- Uses same `musicbrainz.ApplyTypeToggles` logic for consistency
|
||||||
|
|
||||||
|
## Implementation Details
|
||||||
|
|
||||||
|
### Shared Filtering Logic
|
||||||
|
|
||||||
|
Both paths use the same underlying filtering logic:
|
||||||
|
- `musicbrainz.ApplyTypeToggles` for ExternalRelease filtering
|
||||||
|
- `musicbrainz.ApplyTypeTogglesToReleaseGroups` for ReleaseGroup filtering
|
||||||
|
- Consistent criteria:
|
||||||
|
- IgnoreSingles: Type == "Single" OR Type == "EP" OR SecondaryTypes contains "Single"/"EP"
|
||||||
|
- IgnoreCompilations: Type == "Compilation" OR SecondaryTypes contains "Compilation"
|
||||||
|
|
||||||
|
### TypeFilter Structure
|
||||||
|
|
||||||
|
```go
|
||||||
|
type TypeFilter struct {
|
||||||
|
IgnoreSingles bool
|
||||||
|
IgnoreCompilations bool
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Filter Application
|
||||||
|
|
||||||
|
- **MusicBrainz Path**: `ApplyTypeToggles(releases, FilterOptions{IgnoreSingles: s.IgnoreSingles, IgnoreCompilations: s.IgnoreCompilations})`
|
||||||
|
- **Scanner Path**: `filter.suppressed(ext)` which internally uses the same logic
|
||||||
|
|
||||||
|
## Benefits
|
||||||
|
|
||||||
|
1. **Storage Efficiency**: Only desired release types are cached in the database
|
||||||
|
2. **Immediate Responsiveness**: User preference changes take effect in real-time
|
||||||
|
3. **Consistent Behavior**: Both code paths produce identical filtering results
|
||||||
|
4. **Cache Efficiency**: Existing cached data remains useful when preferences change
|
||||||
|
|
||||||
|
## Related Files
|
||||||
|
|
||||||
|
- `internal/scanner/diff.go`: TypeFilter definition and suppressed() method
|
||||||
|
- `internal/scanner/scan.go`: ScanArtist and ScanAll functions
|
||||||
|
- `internal/musicbrainz/sync.go`: SyncArtistDiscography and getArtistFilterOptions
|
||||||
|
- `internal/musicbrainz/filter.go`: ApplyTypeToggles and related filtering functions
|
||||||
|
- `internal/database/artist_settings.go`: GetArtistSettings and GetAllArtistSettings
|
||||||
Reference in New Issue
Block a user