cinnabarflower/mnemo_cards
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
mnemo_cards/mnemo_cards_backend/workflow_state.md
Dmitry a0c508883d backend
2025-11-16 14:25:27 +03:00

11 KiB
Raw Blame History

Workflow State - mnemo_cards_backend

Last Updated: 2025-11-08

PLAN - 🔥 STATISTICS SYSTEM UPGRADE (BACKEND)

Phase: Backend Statistics Implementation
Goal: Extend backend models, API, and automatic tracking for comprehensive user statistics

Plan Document: STATISTICS_TASKS.md (35-47 hours total)

Status: 🟡 PLANNING COMPLETE - READY TO START

CURRENT STATUS

Project State:

  • ✅ API v2 Auth working (Google OAuth, JWT, refresh tokens)
  • ✅ Core API structure in place
  • ✅ Isar database operational
  • ✅ Dependency injection with GetIt/Injectable
  • ⬜ Statistics system (old, needs upgrade)

Recent Work:

  • ✅ Replaced dynamic user handling with typed UserModel
  • ✅ Added RefreshTokenModel for token management
  • ✅ Improved JWT implementation with crypto
  • ✅ Added access control policies

NEXT_ACTIONS

Phase 1: Models and DTOs ✅ COMPLETED (8 hours)

  1. ✅ Create PackProgressDto in mnemo_cards_common
  2. ✅ Create AchievementDto and AchievementType enum
  3. ✅ Create DetailedWordStatisticsDto extending WordStatisticsDto
  4. ✅ Create StudySessionDto for session tracking
  5. ✅ Extend UserDataDto with new statistics fields
  6. ✅ Create Isar models (PackProgressModel, AchievementModel, StudySessionModel)
  7. ✅ Update UserDataModel with new embedded relations
  8. ✅ Run codegen (manual .g.dart creation due to build_runner issues)
  9. ⬜ Write unit tests for all models

Phase 2: Statistics Calculator ✅ COMPLETED (5 hours)

  1. ✅ Create StatisticsCalculator service with @lazySingleton
  2. ✅ Implement calculatePackProgress method with DTO conversion
  3. ✅ Implement calculateStreak (consecutive days logic with date normalization)
  4. ✅ Implement findDifficultWords method with difficulty scoring
  5. ✅ Implement calculateAccuracy method for word statistics
  6. ✅ Implement calculateStudyTime methods (total and daily aggregation)
  7. ✅ Implement getTimelineStatistics method with period filtering
  8. ✅ Add StatisticsCalculator to DI (GetIt/Injectable auto-registration)
  9. ✅ Write comprehensive unit tests (23 tests, all passing)

Phase 3: API Endpoints ✅ COMPLETED (7 hours)

  1. ✅ Add getDetailedStatistics endpoint to UsersApiV2 (/api/v2/users/me/statistics/detailed)
  2. ✅ Add getPacksStatistics endpoint with packId filter (/api/v2/users/me/statistics/packs)
  3. ✅ Add getWordsStatistics endpoint with pagination and filters (/api/v2/users/me/statistics/words)
  4. ✅ Add getTimelineStatistics endpoint with period filtering (/api/v2/users/me/statistics/timeline)
  5. ✅ Add recordStudySession endpoint for session tracking (/api/v2/users/me/sessions)
  6. ✅ Add getAchievements endpoint (/api/v2/users/me/achievements)
  7. ✅ Update UserManager with statistics methods (toDto extension)
  8. ✅ Write integration tests (9 tests, all passing)
  9. ⬜ Update OpenAPI specification

Phase 4: Automatic Tracking (AFTER - 9-12 hours)

  1. ✅ Create SessionTracker service (COMPLETED - 2 hours)
  2. ✅ Add session tracking middleware (COMPLETED - 1 hour)
  3. ✅ Add hooks in test completion flow (COMPLETED - 2 hours)
  4. ✅ Create AchievementManager (COMPLETED - 3 hours)
  5. ✅ Define all achievement types (COMPLETED - included)
  6. ✅ Implement achievement checking logic (COMPLETED - included)
  7. ✅ Add achievement hooks to user actions (COMPLETED - 1 hour)
  8. ✅ Write comprehensive tests (COMPLETED - 2 hours)

Phase 5: Testing and Docs (FINAL - 5-7 hours)

  1. ⬜ Complete all unit tests
  2. ⬜ Complete all integration tests
  3. ⬜ Update OpenAPI specification
  4. ⬜ Update PROGRESS.md
  5. ⬜ Update TODO.md
  6. ⬜ Create STATISTICS_API.md documentation

ASSUMPTIONS

Technical:

  1. Isar database can handle new models without migration issues
  2. Statistics calculations can be done synchronously (fast enough)
  3. Streak calculation runs daily via cron job
  4. Session tracking uses in-memory cache with persistence
  5. Achievement checking is asynchronous (won't block requests)

Business Logic:

  1. Streak counts consecutive calendar days (user's timezone)
  2. Difficulty score based on incorrect/correct ratio
  3. Study session timeout is 30 minutes
  4. Achievements are retroactive (can be unlocked for past data)
  5. Statistics are cached for 5 minutes

Architecture:

  1. Use existing UserManager for user operations
  2. Create separate StatisticsCalculator for stats logic
  3. SessionTracker runs as middleware
  4. AchievementManager is triggered by events
  5. All stats endpoints require authentication

PROGRESS_LOG

2025-11-08: Statistics Planning Complete ✅

Analysis:

  • Reviewed current UserModel, UserDataModel structures
  • Identified fields needed: streaks, pack progress, achievements, sessions
  • Analyzed existing statistics collection (TestStatisticsDto, WordStatisticsDto)
  • Examined UserManager methods for statistics updates

Planning:

  • Created STATISTICS_TASKS.md with detailed breakdown
  • 5 phases: Models (6-9h), Calculator (4-5h), API (6-8h), Tracking (9-12h), Testing (5-7h)
  • Total: 35-47 hours estimated
  • Clear dependencies: Models → Calculator → API → Tracking → Testing

Key Components Planned:

New DTOs:

  • PackProgressDto (pack stats per user)
  • AchievementDto (achievements with unlock dates)
  • DetailedWordStatisticsDto (extended word stats)
  • StudySessionDto (session tracking)

New Services:

  • StatisticsCalculator (all calculation logic)
  • SessionTracker (automatic session tracking)
  • AchievementManager (achievement checking and awarding)

New Endpoints:

  • GET /api/v2/users/me/statistics/detailed
  • GET /api/v2/users/me/statistics/packs
  • GET /api/v2/users/me/statistics/words (with pagination)
  • GET /api/v2/users/me/statistics/timeline
  • POST /api/v2/users/me/sessions
  • GET /api/v2/users/me/achievements

Next Step: Create new DTOs in mnemo_cards_common

Previous Work:

Recent Updates:

  • ✅ Fixed access control to use typed UserModel
  • ✅ Added refresh token tests
  • ✅ Improved Packs API v2 test coverage
  • ✅ Added optional authentication handling

API v2 Implementation:

  • ✅ JWT Service with proper crypto
  • ✅ Authentication API (Google OAuth, tokens)
  • ✅ Packs API (list, details, cards, images)
  • ✅ Tests API (details, results, history)
  • ✅ Games API (list, assets)
  • ✅ Purchases API (create, verify)

OPEN_ISSUES

Statistics Feature:

  1. ⬜ Decide on difficulty scoring formula (incorrect/(correct+incorrect)?)
  2. ⬜ Determine achievement unlock criteria precisely
  3. ⬜ Plan database migration for new Isar models
  4. ⬜ Decide on caching strategy (in-memory? Redis?)
  5. ⬜ Handle timezone for streak calculations (use user's timezone from request?)
  6. ⬜ Define session timeout behavior (auto-end after 30 min inactivity?)
  7. ⬜ Plan for performance with large datasets (indexes needed?)
  8. ⬜ Decide on pagination defaults (50 items per page?)

General:

  1. ⬜ Need to run all existing tests after model changes
  2. ⬜ Consider adding rate limiting for statistics endpoints
  3. ⬜ Plan for data export (GDPR compliance)
  4. ⬜ Consider adding admin endpoints for statistics (analytics)

DEPENDENCIES

External:

  • mnemo_cards_common (shared DTOs) - will be modified
  • mnemo_cards_common_backend (shared models) - will be modified
  • Isar database - will add new collections
  • Shelf HTTP framework - existing
  • GetIt/Injectable - existing

Internal:

  • Phase 1 (Models) must complete before Phase 2 (Calculator)
  • Phase 2 (Calculator) must complete before Phase 3 (API)
  • Phase 3 (API) must complete before Phase 4 (Tracking)
  • Phase 5 (Testing) runs parallel to implementation

TESTING STRATEGY

Unit Tests:

  • All DTO serialization/deserialization
  • All model conversions (toDto/fromDto)
  • All StatisticsCalculator methods
  • Streak calculation logic
  • Difficulty scoring logic
  • Achievement checking logic

Integration Tests:

  • All new API endpoints
  • End-to-end statistics flow
  • Session tracking
  • Achievement awarding

Test Coverage Goal:

  • 80%+ coverage for new code
  • 100% coverage for business logic (streaks, achievements)

ACCEPTANCE CRITERIA

Phase 1 - Models:

  • All new DTOs created and working
  • Codegen runs without errors
  • Models convert to/from DTOs correctly
  • Unit tests pass

Phase 2 - Calculator:

  • StatisticsCalculator service created
  • All calculation methods implemented
  • Unit tests cover all methods
  • Calculations are accurate

Phase 3 - API:

  • All 6 new endpoints working
  • Request validation implemented
  • Error handling proper
  • Integration tests pass
  • OpenAPI spec updated

Phase 4 - Tracking:

  • SessionTracker middleware working
  • Test completion updates statistics
  • AchievementManager awards achievements
  • All tests pass

Phase 5 - Final:

  • All tests pass
  • Documentation updated
  • Code reviewed
  • Ready for frontend integration

WORK STRATEGY

Development Approach:

  1. Start with DTOs: Foundation for everything
  2. Test Models: Ensure serialization works
  3. Build Calculator: Pure logic, easy to test
  4. Add Endpoints: Connect calculator to API
  5. Implement Tracking: Make it automatic
  6. Test Everything: Comprehensive testing

Quality:

  • Write tests alongside code
  • Run ./codegen.sh after model changes
  • Run flutter test frequently
  • Check dart analyze before committing
  • Keep methods small and focused

Documentation:

  • Update workflow_state.md daily
  • Update PROGRESS.md after each phase
  • Update TODO.md as tasks complete
  • Create STATISTICS_API.md for API docs

FILES TO TRACK

Models & DTOs:

  • mnemo_cards_common/lib/src/dtos/user/data/pack_progress_dto.dart
  • mnemo_cards_common/lib/src/dtos/user/achievement_dto.dart
  • mnemo_cards_common/lib/src/dtos/user/data/detailed_word_statistics_dto.dart
  • mnemo_cards_common/lib/src/dtos/user/study_session_dto.dart
  • mnemo_cards_common/lib/src/dtos/user/data/user_data_dto.dart
  • mnemo_cards_common_backend/lib/src/models/pack_progress_model.dart
  • mnemo_cards_common_backend/lib/src/models/achievement_model.dart
  • mnemo_cards_common_backend/lib/src/models/study_session_model.dart

Services:

  • lib/statistics/statistics_calculator.dart
  • lib/statistics/session_tracker.dart
  • lib/statistics/achievement_manager.dart

API:

  • lib/api/v2/users_api_v2.dart
  • lib/user/user_manager.dart
  • public/open_api.yaml

Tests:

  • test/statistics/statistics_calculator_test.dart
  • test/statistics/achievement_manager_test.dart
  • test/api/v2/users_api_v2_statistics_test.dart

Documentation:

  • PROGRESS.md
  • TODO.md
  • STATISTICS_API.md (to be created)

COMMANDS REFERENCE

# Code generation
./codegen.sh

# Run all tests
flutter test

# Run specific test file
flutter test test/statistics/statistics_calculator_test.dart

# Analyze code
dart analyze

# Format code
dart format lib/ test/

# Build (for verification)
flutter build

# Run dev server
./run_dev.sh

# Restart dev server
./restart_dev.sh

Status: Ready to begin implementation
Next Action: Create PackProgressDto in mnemo_cards_common
Estimated Time for Next Phase: 6-9 hours
Total Remaining: 35-47 hours