360 lines
11 KiB
Markdown
360 lines
11 KiB
Markdown
# Workflow State - mnemo_cards_backend
|
|
|
|
**Last Updated:** 2025-11-08
|
|
|
|
---
|
|
|
|
## PLAN - 🔥 STATISTICS SYSTEM UPGRADE (BACKEND)
|
|
|
|
**Phase:** Backend Statistics Implementation
|
|
**Goal:** Extend backend models, API, and automatic tracking for comprehensive user statistics
|
|
|
|
**Plan Document:** `STATISTICS_TASKS.md` (35-47 hours total)
|
|
|
|
**Status:** 🟡 PLANNING COMPLETE - READY TO START
|
|
|
|
---
|
|
|
|
## CURRENT STATUS
|
|
|
|
### Project State:
|
|
- ✅ API v2 Auth working (Google OAuth, JWT, refresh tokens)
|
|
- ✅ Core API structure in place
|
|
- ✅ Isar database operational
|
|
- ✅ Dependency injection with GetIt/Injectable
|
|
- ⬜ Statistics system (old, needs upgrade)
|
|
|
|
### Recent Work:
|
|
- ✅ Replaced dynamic user handling with typed UserModel
|
|
- ✅ Added RefreshTokenModel for token management
|
|
- ✅ Improved JWT implementation with crypto
|
|
- ✅ Added access control policies
|
|
|
|
---
|
|
|
|
## NEXT_ACTIONS
|
|
|
|
### Phase 1: Models and DTOs ✅ COMPLETED (8 hours)
|
|
1. ✅ Create PackProgressDto in mnemo_cards_common
|
|
2. ✅ Create AchievementDto and AchievementType enum
|
|
3. ✅ Create DetailedWordStatisticsDto extending WordStatisticsDto
|
|
4. ✅ Create StudySessionDto for session tracking
|
|
5. ✅ Extend UserDataDto with new statistics fields
|
|
6. ✅ Create Isar models (PackProgressModel, AchievementModel, StudySessionModel)
|
|
7. ✅ Update UserDataModel with new embedded relations
|
|
8. ✅ Run codegen (manual .g.dart creation due to build_runner issues)
|
|
9. ⬜ Write unit tests for all models
|
|
|
|
### Phase 2: Statistics Calculator ✅ COMPLETED (5 hours)
|
|
10. ✅ Create StatisticsCalculator service with @lazySingleton
|
|
11. ✅ Implement calculatePackProgress method with DTO conversion
|
|
12. ✅ Implement calculateStreak (consecutive days logic with date normalization)
|
|
13. ✅ Implement findDifficultWords method with difficulty scoring
|
|
14. ✅ Implement calculateAccuracy method for word statistics
|
|
15. ✅ Implement calculateStudyTime methods (total and daily aggregation)
|
|
16. ✅ Implement getTimelineStatistics method with period filtering
|
|
17. ✅ Add StatisticsCalculator to DI (GetIt/Injectable auto-registration)
|
|
18. ✅ Write comprehensive unit tests (23 tests, all passing)
|
|
|
|
### Phase 3: API Endpoints ✅ COMPLETED (7 hours)
|
|
19. ✅ Add getDetailedStatistics endpoint to UsersApiV2 (/api/v2/users/me/statistics/detailed)
|
|
20. ✅ Add getPacksStatistics endpoint with packId filter (/api/v2/users/me/statistics/packs)
|
|
21. ✅ Add getWordsStatistics endpoint with pagination and filters (/api/v2/users/me/statistics/words)
|
|
22. ✅ Add getTimelineStatistics endpoint with period filtering (/api/v2/users/me/statistics/timeline)
|
|
23. ✅ Add recordStudySession endpoint for session tracking (/api/v2/users/me/sessions)
|
|
24. ✅ Add getAchievements endpoint (/api/v2/users/me/achievements)
|
|
25. ✅ Update UserManager with statistics methods (toDto extension)
|
|
26. ✅ Write integration tests (9 tests, all passing)
|
|
27. ⬜ Update OpenAPI specification
|
|
|
|
### Phase 4: Automatic Tracking (AFTER - 9-12 hours)
|
|
28. ✅ Create SessionTracker service (COMPLETED - 2 hours)
|
|
29. ✅ Add session tracking middleware (COMPLETED - 1 hour)
|
|
30. ✅ Add hooks in test completion flow (COMPLETED - 2 hours)
|
|
31. ✅ Create AchievementManager (COMPLETED - 3 hours)
|
|
32. ✅ Define all achievement types (COMPLETED - included)
|
|
33. ✅ Implement achievement checking logic (COMPLETED - included)
|
|
34. ✅ Add achievement hooks to user actions (COMPLETED - 1 hour)
|
|
35. ✅ Write comprehensive tests (COMPLETED - 2 hours)
|
|
|
|
### Phase 5: Testing and Docs (FINAL - 5-7 hours)
|
|
36. ⬜ Complete all unit tests
|
|
37. ⬜ Complete all integration tests
|
|
38. ⬜ Update OpenAPI specification
|
|
39. ⬜ Update PROGRESS.md
|
|
40. ⬜ Update TODO.md
|
|
41. ⬜ Create STATISTICS_API.md documentation
|
|
|
|
---
|
|
|
|
## ASSUMPTIONS
|
|
|
|
### Technical:
|
|
1. Isar database can handle new models without migration issues
|
|
2. Statistics calculations can be done synchronously (fast enough)
|
|
3. Streak calculation runs daily via cron job
|
|
4. Session tracking uses in-memory cache with persistence
|
|
5. Achievement checking is asynchronous (won't block requests)
|
|
|
|
### Business Logic:
|
|
6. Streak counts consecutive calendar days (user's timezone)
|
|
7. Difficulty score based on incorrect/correct ratio
|
|
8. Study session timeout is 30 minutes
|
|
9. Achievements are retroactive (can be unlocked for past data)
|
|
10. Statistics are cached for 5 minutes
|
|
|
|
### Architecture:
|
|
11. Use existing UserManager for user operations
|
|
12. Create separate StatisticsCalculator for stats logic
|
|
13. SessionTracker runs as middleware
|
|
14. AchievementManager is triggered by events
|
|
15. All stats endpoints require authentication
|
|
|
|
---
|
|
|
|
## PROGRESS_LOG
|
|
|
|
### 2025-11-08: Statistics Planning Complete ✅
|
|
|
|
**Analysis:**
|
|
- Reviewed current UserModel, UserDataModel structures
|
|
- Identified fields needed: streaks, pack progress, achievements, sessions
|
|
- Analyzed existing statistics collection (TestStatisticsDto, WordStatisticsDto)
|
|
- Examined UserManager methods for statistics updates
|
|
|
|
**Planning:**
|
|
- Created STATISTICS_TASKS.md with detailed breakdown
|
|
- 5 phases: Models (6-9h), Calculator (4-5h), API (6-8h), Tracking (9-12h), Testing (5-7h)
|
|
- Total: 35-47 hours estimated
|
|
- Clear dependencies: Models → Calculator → API → Tracking → Testing
|
|
|
|
**Key Components Planned:**
|
|
|
|
**New DTOs:**
|
|
- PackProgressDto (pack stats per user)
|
|
- AchievementDto (achievements with unlock dates)
|
|
- DetailedWordStatisticsDto (extended word stats)
|
|
- StudySessionDto (session tracking)
|
|
|
|
**New Services:**
|
|
- StatisticsCalculator (all calculation logic)
|
|
- SessionTracker (automatic session tracking)
|
|
- AchievementManager (achievement checking and awarding)
|
|
|
|
**New Endpoints:**
|
|
- GET /api/v2/users/me/statistics/detailed
|
|
- GET /api/v2/users/me/statistics/packs
|
|
- GET /api/v2/users/me/statistics/words (with pagination)
|
|
- GET /api/v2/users/me/statistics/timeline
|
|
- POST /api/v2/users/me/sessions
|
|
- GET /api/v2/users/me/achievements
|
|
|
|
**Next Step:** Create new DTOs in mnemo_cards_common
|
|
|
|
---
|
|
|
|
### Previous Work:
|
|
|
|
#### Recent Updates:
|
|
- ✅ Fixed access control to use typed UserModel
|
|
- ✅ Added refresh token tests
|
|
- ✅ Improved Packs API v2 test coverage
|
|
- ✅ Added optional authentication handling
|
|
|
|
#### API v2 Implementation:
|
|
- ✅ JWT Service with proper crypto
|
|
- ✅ Authentication API (Google OAuth, tokens)
|
|
- ✅ Packs API (list, details, cards, images)
|
|
- ✅ Tests API (details, results, history)
|
|
- ✅ Games API (list, assets)
|
|
- ✅ Purchases API (create, verify)
|
|
|
|
---
|
|
|
|
## OPEN_ISSUES
|
|
|
|
### Statistics Feature:
|
|
1. ⬜ Decide on difficulty scoring formula (incorrect/(correct+incorrect)?)
|
|
2. ⬜ Determine achievement unlock criteria precisely
|
|
3. ⬜ Plan database migration for new Isar models
|
|
4. ⬜ Decide on caching strategy (in-memory? Redis?)
|
|
5. ⬜ Handle timezone for streak calculations (use user's timezone from request?)
|
|
6. ⬜ Define session timeout behavior (auto-end after 30 min inactivity?)
|
|
7. ⬜ Plan for performance with large datasets (indexes needed?)
|
|
8. ⬜ Decide on pagination defaults (50 items per page?)
|
|
|
|
### General:
|
|
9. ⬜ Need to run all existing tests after model changes
|
|
10. ⬜ Consider adding rate limiting for statistics endpoints
|
|
11. ⬜ Plan for data export (GDPR compliance)
|
|
12. ⬜ Consider adding admin endpoints for statistics (analytics)
|
|
|
|
---
|
|
|
|
## DEPENDENCIES
|
|
|
|
### External:
|
|
- mnemo_cards_common (shared DTOs) - will be modified
|
|
- mnemo_cards_common_backend (shared models) - will be modified
|
|
- Isar database - will add new collections
|
|
- Shelf HTTP framework - existing
|
|
- GetIt/Injectable - existing
|
|
|
|
### Internal:
|
|
- Phase 1 (Models) must complete before Phase 2 (Calculator)
|
|
- Phase 2 (Calculator) must complete before Phase 3 (API)
|
|
- Phase 3 (API) must complete before Phase 4 (Tracking)
|
|
- Phase 5 (Testing) runs parallel to implementation
|
|
|
|
---
|
|
|
|
## TESTING STRATEGY
|
|
|
|
### Unit Tests:
|
|
- All DTO serialization/deserialization
|
|
- All model conversions (toDto/fromDto)
|
|
- All StatisticsCalculator methods
|
|
- Streak calculation logic
|
|
- Difficulty scoring logic
|
|
- Achievement checking logic
|
|
|
|
### Integration Tests:
|
|
- All new API endpoints
|
|
- End-to-end statistics flow
|
|
- Session tracking
|
|
- Achievement awarding
|
|
|
|
### Test Coverage Goal:
|
|
- 80%+ coverage for new code
|
|
- 100% coverage for business logic (streaks, achievements)
|
|
|
|
---
|
|
|
|
## ACCEPTANCE CRITERIA
|
|
|
|
### Phase 1 - Models:
|
|
- [ ] All new DTOs created and working
|
|
- [ ] Codegen runs without errors
|
|
- [ ] Models convert to/from DTOs correctly
|
|
- [ ] Unit tests pass
|
|
|
|
### Phase 2 - Calculator:
|
|
- [ ] StatisticsCalculator service created
|
|
- [ ] All calculation methods implemented
|
|
- [ ] Unit tests cover all methods
|
|
- [ ] Calculations are accurate
|
|
|
|
### Phase 3 - API:
|
|
- [ ] All 6 new endpoints working
|
|
- [ ] Request validation implemented
|
|
- [ ] Error handling proper
|
|
- [ ] Integration tests pass
|
|
- [ ] OpenAPI spec updated
|
|
|
|
### Phase 4 - Tracking:
|
|
- [ ] SessionTracker middleware working
|
|
- [ ] Test completion updates statistics
|
|
- [ ] AchievementManager awards achievements
|
|
- [ ] All tests pass
|
|
|
|
### Phase 5 - Final:
|
|
- [ ] All tests pass
|
|
- [ ] Documentation updated
|
|
- [ ] Code reviewed
|
|
- [ ] Ready for frontend integration
|
|
|
|
---
|
|
|
|
## WORK STRATEGY
|
|
|
|
### Development Approach:
|
|
1. **Start with DTOs:** Foundation for everything
|
|
2. **Test Models:** Ensure serialization works
|
|
3. **Build Calculator:** Pure logic, easy to test
|
|
4. **Add Endpoints:** Connect calculator to API
|
|
5. **Implement Tracking:** Make it automatic
|
|
6. **Test Everything:** Comprehensive testing
|
|
|
|
### Quality:
|
|
- Write tests alongside code
|
|
- Run `./codegen.sh` after model changes
|
|
- Run `flutter test` frequently
|
|
- Check `dart analyze` before committing
|
|
- Keep methods small and focused
|
|
|
|
### Documentation:
|
|
- Update workflow_state.md daily
|
|
- Update PROGRESS.md after each phase
|
|
- Update TODO.md as tasks complete
|
|
- Create STATISTICS_API.md for API docs
|
|
|
|
---
|
|
|
|
## FILES TO TRACK
|
|
|
|
**Models & DTOs:**
|
|
- `mnemo_cards_common/lib/src/dtos/user/data/pack_progress_dto.dart`
|
|
- `mnemo_cards_common/lib/src/dtos/user/achievement_dto.dart`
|
|
- `mnemo_cards_common/lib/src/dtos/user/data/detailed_word_statistics_dto.dart`
|
|
- `mnemo_cards_common/lib/src/dtos/user/study_session_dto.dart`
|
|
- `mnemo_cards_common/lib/src/dtos/user/data/user_data_dto.dart`
|
|
- `mnemo_cards_common_backend/lib/src/models/pack_progress_model.dart`
|
|
- `mnemo_cards_common_backend/lib/src/models/achievement_model.dart`
|
|
- `mnemo_cards_common_backend/lib/src/models/study_session_model.dart`
|
|
|
|
**Services:**
|
|
- `lib/statistics/statistics_calculator.dart`
|
|
- `lib/statistics/session_tracker.dart`
|
|
- `lib/statistics/achievement_manager.dart`
|
|
|
|
**API:**
|
|
- `lib/api/v2/users_api_v2.dart`
|
|
- `lib/user/user_manager.dart`
|
|
- `public/open_api.yaml`
|
|
|
|
**Tests:**
|
|
- `test/statistics/statistics_calculator_test.dart`
|
|
- `test/statistics/achievement_manager_test.dart`
|
|
- `test/api/v2/users_api_v2_statistics_test.dart`
|
|
|
|
**Documentation:**
|
|
- `PROGRESS.md`
|
|
- `TODO.md`
|
|
- `STATISTICS_API.md` (to be created)
|
|
|
|
---
|
|
|
|
## COMMANDS REFERENCE
|
|
|
|
```bash
|
|
# Code generation
|
|
./codegen.sh
|
|
|
|
# Run all tests
|
|
flutter test
|
|
|
|
# Run specific test file
|
|
flutter test test/statistics/statistics_calculator_test.dart
|
|
|
|
# Analyze code
|
|
dart analyze
|
|
|
|
# Format code
|
|
dart format lib/ test/
|
|
|
|
# Build (for verification)
|
|
flutter build
|
|
|
|
# Run dev server
|
|
./run_dev.sh
|
|
|
|
# Restart dev server
|
|
./restart_dev.sh
|
|
```
|
|
|
|
---
|
|
|
|
**Status:** Ready to begin implementation
|
|
**Next Action:** Create PackProgressDto in mnemo_cards_common
|
|
**Estimated Time for Next Phase:** 6-9 hours
|
|
**Total Remaining:** 35-47 hours
|