mnemo_cards/PROGRESS.md

2025-12-07
- Добавлена подробная инструкция по миграции в Coolify (Docker backend, статический web/admin, внешняя БД) — `docs/coolify-migration.md`.
# Mnemo Cards Project Progress

## Infrastructure & Deployment

### ✅ Completed Tasks

- **nginx Configuration System**: Implemented robust nginx configuration generation and deployment
  - Created `generate-nginx-configs.sh` for generating all nginx configurations
  - Created `deploy-nginx-configs.sh` for safe deployment with validation
  - Updated `check-nginx.sh` with port conflict detection and cleanup
  - Integrated into CI/CD pipeline

- **Reverse Proxy Architecture**: Properly configured nginx as reverse proxy for all services
  - `mnemo-cards.online` → Flutter web app (static files)
  - `code.mnemo-cards.online` → Forgejo (port 3000)
  - `vscode.mnemo-cards.online` → VSCode Server (port 8443)

- **Automated Version Management**: Implemented pre-deploy version bumping
  - Created `bump-version.sh` script for automatic version increment
  - Integrated as pre-hook in CI/CD pipeline
  - Updates both pubspec.yaml and API configuration
  - Automatic git commit with version changes

- **SSL/TLS Configuration**: Secure HTTPS setup with Let's Encrypt certificates
  - Multi-domain certificates for all subdomains
  - Proper certificate permissions and renewal

- **Admin API Configuration**: Fixed admin authentication routing
  - Admin panel now correctly routes authentication requests to production backend
  - Prevents localhost:8000 routing issues during development
  - Separate API client for admin endpoints ensures proper server targeting

- **Admin Cards: Voice Upload on Save**: Fixed voice upload when updating/creating cards in admin
  - Added "Voice (will be added on Save)" to card editor and upload on Create/Update
  - Ensured voices list still loads while the form is disabled during save
  - Added unit tests for the upsert+voice flow (`mnemo_cards_admin`)
- **Admin Packs: Color Palette Picker**: Added palette-based color selection for packs
  - Replaced free-text color input with native color picker (palette)
  - Removed hardcoded default swatch colors (keep only palette)
  - Added unit tests for the new color picker component (`mnemo_cards_admin`)
- **Admin Tests: Pack Linking**: Added ability to link tests to packs in admin
  - Test editor now supports selecting packs and syncing links on Save
  - Added unit test coverage for the new pack-linking component (`mnemo_cards_admin`)
- **Admin Tests: Color Picker + Pack Color Sync**: Added palette-based test color selection
  - Test editor now uses the same palette picker as packs
  - When selecting/changing a pack in test editor, test color auto-updates to the pack color
- **Admin Bulk Card Editor: Fix pack mapping typing**: Fixed TypeScript build error in pack list loading
  - Removed unsafe `Record<string, unknown>` cast for `CardPackPreviewDto`
  - Added unit test and JSDOM polyfills for Radix Select in tests (`mnemo_cards_admin`)

- **Service Reliability**: Enhanced service management and monitoring
  - Port conflict detection and resolution
  - Orphaned process cleanup
  - Configuration validation before deployment
  - Emergency fallback configurations
  - Coolify container orchestration platform deployed

### 🔄 In Progress

- **Coolify Migration**: Container orchestration platform for services
  - Coolify installed and configured on `coolify.mnemo-cards.online`
  - Automatic startup configured via cron @reboot
  - Ready for service migration (backend, web, admin)
- **Monitoring & Alerting**: Implement comprehensive service monitoring
- **Backup Automation**: Automated backup procedures for all services

### 📋 Planned Tasks

- **Load Balancing**: Implement load balancing for high availability
- **CDN Integration**: Content delivery network for static assets
- **Security Hardening**: Additional security measures and hardening

## Development Status

### Web Application (mnemo_cards_web_v2)
- **Framework**: Flutter Web
- **Status**: Active development
- **Key Features**: Vocabulary learning, testing, statistics
- **Recent Updates**:
  - Added card voice support (metadata, API, web playback with error UI)
    - New voices collection linked to cards and protected endpoints for list/mp3 (Bearer)
    - Web playback: download bytes via authenticated API and play from memory (fixes 401 on `<audio src>` without headers)
    - Added unit tests for voice metadata fetching and voice-bytes download
  - Card UI: moved voice play icon into a `Stack` overlay so it doesn't take an extra row on the card (with widget test coverage)
  - CardViewer UI: navigation arrows are constrained under the card on wide screens; favorites (heart) is now part of the card (top-right); voice controls moved to top-left (header text stays centered)
    - Added widget test coverage for wide-screen navigation + header control positioning
  - Pack details: "Перемешать" button no longer has an active (pressed) state (widget test updated)
  - Pack details: Control buttons (Сетка/Карточки, Перемешать, Избранные) now hide text when there's not enough space, showing only icons (widget tests added)
  - Pack details: removed the "view" (eye) icon on cards; moved the favorite (heart) icon to the top-right corner (list + grid) (widget test updated)
  - Added app version display on authentication page
  - **UI Components Refactoring**: Extracted authentication components
    - Created `SignInWithGoogleButton` component for Google authentication
    - Created `SignInWithTelegram` component for Telegram authentication
    - Improved code maintainability and reusability
  - **Profile Page UI/UX Improvements**: Enhanced profile page design and functionality
    - Removed guest view - profile now only accessible for authenticated users
    - Removed account statistics (packs/purchases) from profile
    - Removed language toggle from settings
    - Created custom `ThemeToggleWidget` with sun/moon icons and animations
    - Enhanced subscription section with premium badge and upgrade button for free users
    - Added staggered animations for card appearances
    - Improved logout flow with confirmation dialog
    - Added comprehensive unit tests for new components
- **Game/Test Flow Cleanup**: Simplified testing UX and fixed theme visuals
  - Removed legacy traditional test entry from `test_page.dart` (interactive mode only)
  - Added SafeArea wrapping and themed question cards on `game_page.dart`
  - Added widget tests for themed question cards, exit control, and interactive-only notice
  - Timer/progress now sourced from live session state on `game_page.dart`
  - Fixed broken game navigation: `TestPage` now pushes `/game/:id` so Exit/Finish/Back work (incl. deep-link fallback) + widget tests
- **Game Tests Package**: Started shared package `games/packages/game_tests`
  - Added configurable `GameTestSettings` (sounds/haptics/delay)
  - Integrated into `mnemo_cards_web_v2` via `TestsModule`/`TestsStateManager`
  - **Test Page Code Quality Fix**: Fixed critical code duplication and compilation issues
    - Removed 700+ lines of duplicate code in `test_page.dart`
    - Added missing state variables for test functionality
    - Created `TestResult` class for proper result handling
    - Fixed deprecated `withOpacity` method calls
    - Restored full test functionality with interactive game and traditional test modes
  - **PackDetailsHeader Code Fix**: Fixed critical structural issues in pack details header widget
    - Resolved broken method structure with hanging code blocks
    - Created missing `_buildTitleSection` method
    - Removed undefined `cachedImage` variable usage
    - Cleaned up unused imports
    - Fixed all linter errors and compilation issues
  - **Matrix Test (image selection)**: Added new multi-stage matrix question type (one screen)
    - Backend generator picks cards from pack pool and stores `matrixSize` in `uiData`
    - Web UI: image grid + target `original` word; wrong = shake + error sound; correct = flip to translation then disappear
    - Admin: question type selector + matrix size form; JSON (de)serialization + vitest coverage
  - **CardViewer Carousel Looping**: Implemented infinite carousel scrolling for card viewer
    - Cards now loop seamlessly: last card → first card and first card → last card
    - Works for both swipe gestures and navigation buttons
    - Added comprehensive unit tests for looping functionality
  - **Loading Screens Enhancement**: Replaced basic loading screens with themed images
    - Removed CircularProgressIndicator as requested
    - Added sun image (el_sol.png) for light theme and moon image (la_luna.png) for dark theme
    - Updated both initialization placeholder and app loading screens to use theme-aware images
    - Added assets/images/ to pubspec.yaml for image loading

### Mobile Application (mnemo_cards)
- **Framework**: Flutter
- **Status**: Active development
- **Key Features**: Cross-platform mobile app

### Backend (mnemo_cards_backend)
- **Framework**: Dart with Shelf
- **Status**: Production ready
- **Key Features**: API, authentication, data management
- **Recent Updates**:
  - **Card Voices: Stop Returning base64**: `/api/v2/packs/.../voices` now returns a direct `url` to audio
    - Admin voice uploads are persisted to `data/voice/` and DB stores only a file name (no base64)
    - Public voice list heals legacy base64-in-DB by migrating to file on first read
    - `/api/v2/voice/<voiceId>` serves bytes (or redirects for remote URLs)
  - **Admin Tests: Stop Returning base64 Images**: `/api/v2/admin/tests/<id>` now returns image **links**, never base64
    - Self-heals legacy base64/data-url values on read by persisting them into `data/cards/` and storing only a `cardId` in DB
    - Normalizes admin round-trip values (`/api/.../cards/<id>/image` → `<cardId>`) so editing tests doesn't pollute DB with API URLs
    - When linking a test to a pack, automatically links all referenced image cards to that pack so `/api/v2/packs/<packId>/cards/<cardId>/image` works
    - Added focused backend unit test: `mnemo_cards_backend/test/api/v2/admin_tests_api_v2_image_urls_test.dart`
  - **User Telegram Field**: Added telegram field to user model
    - Added `telegram` field to `UserModel` and `UserDto` alongside `email`
    - Added `telegram` column to `users` table in database (migration v2→v3)
    - Updated all authentication flows (Google OAuth, Telegram Web App, Telegram bot)
    - Telegram usernames now stored in `telegram` field instead of `email`
    - Added comprehensive unit tests for UserModel and UserDto telegram field
    - Migration automatically moves telegram usernames from email to telegram field
  - **Generated Tests Cleanup**: Fixed generated tests accumulating over time
    - Fixed linking bug: generated tests were created in DB but not linked to packs (caused orphan tests)
    - Added cron cleanup to purge orphan `version='generated'` tests and hard-delete old soft-deleted generated tests (TTL)
    - Added DAO helpers and a focused unit test for the cleanup logic
    - Added regression test to ensure `updateGeneratedTests()` creates a pack link
  - **Matrix Test (image selection)**: Added matrix question generator + API storage support (cards auto-generated if missing)
  - **MinIO Migration**: Migrated file storage from local filesystem to MinIO object storage
    - Created MinioService for MinIO operations (upload, presigned URLs, delete, fileExists)
    - Created MediaApiV2 with endpoints for file uploads (card-image, test-image, voice) and presigned URL generation
    - Updated AdminCardsApiV2 and AdminTestsApiV2 to upload files to MinIO instead of local disk
    - Updated PacksApiV2 and TestManager to generate presigned URLs and embed them in DTOs
    - Updated DTOs (GameCardDto, VoiceDto, TestDto) to include presigned URL fields
    - Added unit tests for MinioService and MediaApiV2
    - **Admin Panel Updates**: Updated admin panel components for MinIO integration
      - Updated ImageUpload component: uploads via Media API, saves objectId, previews via presigned URLs
      - Updated AudioUpload component: uploads via Media API, saves objectId
      - Updated BulkCardUpload: batch uploads via Media API with progress indicators
      - Updated preview logic: uses presigned URLs from backend when available
      - Created Media API client (mediaApi) for file uploads
    - **Web App Updates**: Updated web app for MinIO integration
      - Updated DTOs in mnemo_cards_common: added imageUrl/imageBackUrl/presignedUrl fields to all relevant DTOs
      - Updated test question DTOs: added imageUrl to SimpleTestQuestionBody, InputButtonsTestQuestionBody, TestButtonDto, MatrixCardDto
      - Updated backend test_manager: adds imageUrl while keeping image (objectId) for questions and buttons
      - Updated all card display widgets: card_viewer, card_flipper, pack_details_page, pack_card_item to use imageUrl from DTO
      - Updated test widgets: matrix_widget, answer_options, question_display to use presigned URLs
      - Updated tests_state_manager: converts TestButtonDto and MatrixCardDto to use imageUrl when available
      - All widgets maintain backward compatibility with fallback to ApiConfigV2 URL building
    - **Test Images Fix**: Fixed missing images in game tests
      - Fixed matrix question generation: now uses card.image instead of card.id for images
      - Unified MinIO bucket structure: all images (cards and tests) now use cardImagesBucket
      - Fixed image conversion logic: keeps image as objectId and adds imageUrl as presigned URL
      - Removed testImagesBucket - simplified to single bucket for all images
      - Updated all API endpoints and tests to use unified bucket structure
      - Added comprehensive unit tests for image conversion in test_manager_image_conversion_test.dart

### Common Libraries
- **mnemo_cards_common**: Shared models and utilities
- **mnemo_cards_common_backend**: Backend-specific common code
- **mnemo_cards_frontend_common**: Frontend-specific common code

## Testing

### Unit Tests
- **Coverage**: Basic test coverage implemented
- **Status**: Ongoing - needs expansion

### Integration Tests
- **Status**: Minimal - needs implementation

### E2E Tests
- **Status**: Not implemented

## Deployment Pipeline

### CI/CD Status
- **Platform**: Forgejo Actions (self-hosted)
- **Triggers**: Push to master branch
- **Jobs**:
  - Backend deployment
  - Web application deployment
  - Admin panel deployment (improved portability)
  - Telegram bot deployment
  - Nginx configuration deployment
  - Service verification

- **Deployment Script Improvements**:
  - Enhanced admin panel deployment script portability
  - Script can now be run from any directory within the project
  - Automatic project root detection and navigation
  - Improved error handling and user feedback

### Environments
- **Production**: `147.45.152.129` (mnemo-cards.online)
- **Development**: Local development environments

## Performance Metrics

### Target Metrics
- **Page Load Time**: < 3 seconds
- **API Response Time**: < 500ms
- **Uptime**: 99.9%

### Current Status
- Basic monitoring in place
- Performance optimization ongoing

## Security

### Implemented
- HTTPS enforcement
- SSL/TLS certificates (Let's Encrypt for all domains including admin.mnemo-cards.online)
- Automated SSL certificate renewal
- Basic firewall rules
- Service isolation

### Planned
- Advanced security headers
- Rate limiting
- Intrusion detection
- Regular security audits

---

*Last updated: December 19, 2025*