mnemo_cards/mnemo_cards_backend/STATISTICS_TASKS.md

# Statistics Backend Tasks

**Project:** mnemo_cards_backend  
**Feature:** Statistics System Upgrade  
**Created:** 2025-11-08

---

## Phase 1: Backend - Models and DTOs

### Task 1.1: Create New DTOs in mnemo_cards_common ✅ PRIORITY

**Estimated Time:** 3-4 hours

**Files to Create:**

1. `mnemo_cards_common/lib/src/dtos/user/data/pack_progress_dto.dart`
```dart
@JsonSerializable()
@CopyWith()
class PackProgressDto {
  final String packId;
  final int totalCards;
  final int learnedCards;
  final int studyTimeMinutes;
  final DateTime? lastStudyDate;
  final DateTime? firstStudyDate;
  final Map<String, int> cardAttempts;
  final double averageAccuracy;
}
```

2. `mnemo_cards_common/lib/src/dtos/user/achievement_dto.dart`
```dart
@JsonSerializable()
@CopyWith()
class AchievementDto {
  final String id;
  final String title;
  final String description;
  final String? iconUrl;
  final DateTime? unlockedAt;
  final AchievementType type;
  final double progress; // 0.0 to 1.0
}

enum AchievementType {
  firstSteps,
  streak,
  wordsMaster,
  perfectScore,
  speedLearner,
  dedicated,
}
```

3. `mnemo_cards_common/lib/src/dtos/user/data/detailed_word_statistics_dto.dart`
```dart
@JsonSerializable()
@CopyWith()
class DetailedWordStatisticsDto {
  final String word;
  final double correct;
  final double incorrect;
  final double skipped;
  final Set<TestQuestionType> questionTypes;
  final DateTime? lastReviewed;
  final DateTime? firstLearned;
  final double difficultyScore; // 0.0 to 1.0
  final bool needsReview;
  final String? packId;
}
```

4. `mnemo_cards_common/lib/src/dtos/user/study_session_dto.dart`
```dart
@JsonSerializable()
@CopyWith()
class StudySessionDto {
  final String? sessionId;
  final DateTime startTime;
  final DateTime? endTime;
  final int wordsLearned;
  final int testsCompleted;
  final double accuracy;
  final String? packId;
  final String? testId;
}
```

**Steps:**
- [ ] Create PackProgressDto with all fields
- [ ] Create AchievementDto and AchievementType enum
- [ ] Create DetailedWordStatisticsDto extending WordStatisticsDto
- [ ] Create StudySessionDto
- [ ] Run `./codegen.sh` to generate .g.dart files
- [ ] Export all new DTOs in main export file
- [ ] Write unit tests for DTO serialization/deserialization

---

### Task 1.2: Extend UserDataDto

**Estimated Time:** 1-2 hours

**File:** `mnemo_cards_common/lib/src/dtos/user/data/user_data_dto.dart`

**Add Fields:**
```dart
@JsonSerializable()
@CopyWith()
class UserDataDto {
  // Existing
  final AllWordsStatisticsDto? allWordsStatistics;
  final AllTestsStatisticsDto? allTestsStatistics;
  
  // NEW FIELDS
  final DateTime? lastTimeOnline;
  final int totalStudyTimeMinutes;
  final int currentStreak;
  final int longestStreak;
  final Map<String, PackProgressDto> packProgress;
  final List<DateTime> studyDates; // for streak calculation
  final Map<String, int> categoryMinutes; // category -> minutes
  final List<AchievementDto> achievements;
}
```

**Steps:**
- [ ] Add new fields to UserDataDto
- [ ] Update copyWith to include new fields
- [ ] Run codegen
- [ ] Update tests

---

### Task 1.3: Create Backend Isar Models

**Estimated Time:** 2-3 hours

**Files to Create:**

1. `mnemo_cards_common_backend/lib/src/models/pack_progress_model.dart`
2. `mnemo_cards_common_backend/lib/src/models/achievement_model.dart`
3. `mnemo_cards_common_backend/lib/src/models/study_session_model.dart`

**Steps:**
- [ ] Create Isar models corresponding to DTOs
- [ ] Add toDto() methods
- [ ] Add fromDto() methods
- [ ] Update UserDataModel to include new relations
- [ ] Run codegen
- [ ] Write unit tests for model conversions

---

## Phase 2: Backend - API Endpoints

### Task 2.1: Create StatisticsCalculator Service

**Estimated Time:** 4-5 hours

**File:** `mnemo_cards_backend/lib/statistics/statistics_calculator.dart` (new)

**Methods:**
```dart
@lazySingleton
class StatisticsCalculator {
  /// Calculate pack progress for user
  Future<PackProgressDto> calculatePackProgress(
    UserModel user, 
    String packId,
  );
  
  /// Calculate current streak
  int calculateStreak(List<DateTime> studyDates);
  
  /// Find difficult words that need review
  List<DetailedWordStatisticsDto> findDifficultWords(
    UserDataModel data, 
    {int limit = 20}
  );
  
  /// Calculate overall accuracy
  double calculateAccuracy(AllWordsStatisticsDto stats);
  
  /// Calculate daily study time
  Map<DateTime, int> calculateDailyStudyTime(
    List<StudySessionModel> sessions,
  );
  
  /// Calculate total study time
  int calculateTotalStudyTime(UserDataModel data);
  
  /// Get timeline statistics
  Map<String, dynamic> getTimelineStatistics(
    UserDataModel data,
    {required String period, DateTime? from, DateTime? to}
  );
}
```

**Steps:**
- [ ] Create StatisticsCalculator class
- [ ] Implement calculatePackProgress
- [ ] Implement calculateStreak (consecutive days logic)
- [ ] Implement findDifficultWords (based on accuracy)
- [ ] Implement calculateAccuracy
- [ ] Implement calculateDailyStudyTime
- [ ] Implement calculateTotalStudyTime
- [ ] Implement getTimelineStatistics
- [ ] Add to DI
- [ ] Write comprehensive unit tests

---

### Task 2.2: Add Statistics Endpoints to UsersApiV2

**Estimated Time:** 4-5 hours

**File:** `mnemo_cards_backend/lib/api/v2/users_api_v2.dart`

**New Endpoints:**

```dart
/// GET /api/v2/users/me/statistics/detailed
/// Returns detailed user statistics
@Route.get('/users/me/statistics/detailed')
Future<Response> getDetailedStatistics(Request request);

/// GET /api/v2/users/me/statistics/packs
/// Returns statistics for all packs or specific pack
/// Query: ?packId=xxx
@Route.get('/users/me/statistics/packs')
Future<Response> getPacksStatistics(Request request);

/// GET /api/v2/users/me/statistics/words
/// Returns paginated word statistics
/// Query: ?packId=xxx&limit=50&offset=0&sortBy=difficulty&needsReview=true
@Route.get('/users/me/statistics/words')
Future<Response> getWordsStatistics(Request request);

/// GET /api/v2/users/me/statistics/timeline
/// Returns timeline statistics
/// Query: ?period=week&from=2024-01-01&to=2024-12-31
@Route.get('/users/me/statistics/timeline')
Future<Response> getTimelineStatistics(Request request);

/// POST /api/v2/users/me/sessions
/// Start or end study session
@Route.post('/users/me/sessions')
Future<Response> recordStudySession(Request request);

/// GET /api/v2/users/me/achievements
/// Returns user achievements
@Route.get('/users/me/achievements')
Future<Response> getAchievements(Request request);
```

**Steps:**
- [ ] Add getDetailedStatistics endpoint
- [ ] Add getPacksStatistics endpoint with optional packId filter
- [ ] Add getWordsStatistics endpoint with pagination and filters
- [ ] Add getTimelineStatistics endpoint
- [ ] Add recordStudySession endpoint
- [ ] Add getAchievements endpoint
- [ ] Implement proper error handling
- [ ] Add request validation
- [ ] Write integration tests for all endpoints

---

### Task 2.3: Update UserManager with Statistics Methods

**Estimated Time:** 2-3 hours

**File:** `mnemo_cards_backend/lib/user/user_manager.dart`

**New Methods:**
```dart
class UserManager {
  final StatisticsCalculator _statsCalculator;
  
  // Existing methods...
  
  /// Get detailed user statistics
  Future<UserDataDto> getDetailedStatistics(UserModel user);
  
  /// Get pack statistics
  Future<List<PackProgressDto>> getPacksStatistics(
    UserModel user, 
    {String? packId}
  );
  
  /// Get word statistics with pagination
  Future<Map<String, dynamic>> getWordsStatistics(
    UserModel user, {
    String? packId,
    int limit = 50,
    int offset = 0,
    String sortBy = 'difficulty',
    bool needsReview = false,
  });
  
  /// Record study session
  Future<void> recordStudySession(
    UserModel user, 
    StudySessionDto session,
  );
  
  /// Update user streak
  Future<void> updateStreak(UserModel user);
}
```

**Steps:**
- [ ] Add StatisticsCalculator to constructor
- [ ] Implement getDetailedStatistics
- [ ] Implement getPacksStatistics with optional filtering
- [ ] Implement getWordsStatistics with pagination/sorting
- [ ] Implement recordStudySession
- [ ] Implement updateStreak (call daily)
- [ ] Write unit tests

---

## Phase 3: Backend - Automatic Tracking

### Task 3.1: Create Session Tracking Middleware

**Estimated Time:** 3-4 hours

**File:** `mnemo_cards_backend/lib/statistics/session_tracker.dart` (new)

**Features:**
- Track when user starts/ends session
- Auto-update lastTimeOnline
- Calculate session duration
- Store session in database

**Steps:**
- [ ] Create SessionTracker class
- [ ] Add session start/end logic
- [ ] Integrate with existing auth middleware
- [ ] Store active sessions in memory (with TTL)
- [ ] Auto-cleanup expired sessions
- [ ] Write unit tests

---

### Task 3.2: Add Hooks for Statistics Updates

**Estimated Time:** 2-3 hours

**Files to Modify:**
- `mnemo_cards_backend/lib/api/v2/users_api_v2.dart` (addUserTestStatistics)
- `mnemo_cards_backend/lib/user/user_manager.dart` (addTestStatistics)

**Add After Test Completion:**
- Update pack progress
- Update streak (if needed)
- Check and award achievements
- Update total study time

**Steps:**
- [ ] Add hook in addTestStatistics
- [ ] Call statistics calculator
- [ ] Update pack progress
- [ ] Update streak
- [ ] Trigger achievement check
- [ ] Write tests

---

### Task 3.3: Create Achievement Manager

**Estimated Time:** 4-5 hours

**File:** `mnemo_cards_backend/lib/statistics/achievement_manager.dart` (new)

**Features:**
```dart
@lazySingleton
class AchievementManager {
  /// Check and award achievements after action
  Future<List<AchievementDto>> checkAchievements(UserModel user);
  
  /// Check specific achievement
  Future<AchievementDto?> checkAchievement(
    UserModel user, 
    AchievementType type,
  );
  
  /// Award achievement
  Future<void> awardAchievement(
    UserModel user, 
    AchievementDto achievement,
  );
  
  /// Get all possible achievements
  List<AchievementDto> getAllAchievements();
}
```

**Achievement Types:**
- First word learned
- First test completed
- First pack completed
- Streak milestones (3, 7, 30, 100 days)
- Words milestones (10, 50, 100, 500, 1000)
- Perfect test score
- Speed learner
- Night owl / Early bird
- Total study time milestones

**Steps:**
- [ ] Create AchievementManager
- [ ] Define all achievement types
- [ ] Implement check logic for each type
- [ ] Implement award logic
- [ ] Add to DI
- [ ] Write comprehensive tests

---

## Phase 4: Testing

### Task 4.1: Unit Tests

**Estimated Time:** 3-4 hours

**Test Files:**
- `test/statistics/statistics_calculator_test.dart`
- `test/statistics/achievement_manager_test.dart`
- `test/statistics/session_tracker_test.dart`
- `test/user/user_manager_statistics_test.dart`

**Coverage:**
- All StatisticsCalculator methods
- Achievement checking logic
- Session tracking
- DTO conversions
- Streak calculations
- Difficulty calculations

---

### Task 4.2: Integration Tests

**Estimated Time:** 2-3 hours

**Test File:** `test/api/v2/users_api_v2_statistics_test.dart`

**Tests:**
- GET /users/me/statistics/detailed
- GET /users/me/statistics/packs
- GET /users/me/statistics/words (with filters)
- GET /users/me/statistics/timeline
- POST /users/me/sessions
- GET /users/me/achievements

---

## Phase 5: Documentation

### Task 5.1: Update OpenAPI Spec

**Estimated Time:** 1-2 hours

**File:** `mnemo_cards_backend/public/open_api.yaml`

**Add:**
- All new statistics endpoints
- Request/response schemas
- Query parameters
- Examples

---

### Task 5.2: Update Documentation

**Estimated Time:** 1 hour

**Files:**
- Update `PROGRESS.md`
- Update `TODO.md`
- Create `STATISTICS_API.md` with API documentation

---

## Summary

**Total Estimated Time:** 35-47 hours

**Priority Order:**
1. Task 1.1, 1.2, 1.3 - Models and DTOs (6-9 hours)
2. Task 2.1 - StatisticsCalculator (4-5 hours)
3. Task 2.2, 2.3 - API Endpoints (6-8 hours)
4. Task 3.1, 3.2, 3.3 - Auto Tracking (9-12 hours)
5. Task 4.1, 4.2 - Testing (5-7 hours)
6. Task 5.1, 5.2 - Documentation (2-3 hours)

**Dependencies:**
- Tasks 1.x must be done first
- Tasks 2.x depend on 1.x
- Tasks 3.x depend on 2.x
- Tasks 4.x can be done in parallel with development
- Tasks 5.x should be done last

---

**Start Date:** TBD  
**Target Completion:** TBD