Backend Stability, Basically functional.

2025-10-27 12:38:22 +08:00
parent d0d3373b3b
commit 4ea30cc12e
59 changed files with 5034 additions and 35 deletions
--- a/spring-backend/IMPLEMENTATION_STATUS.md
+++ b/spring-backend/IMPLEMENTATION_STATUS.md
@@ -0,0 +1,342 @@
+# Group Ironmen Backend - Implementation Status
+
+## Project Overview
+
+**Goal**: Migrate group-ironmen-master Rust backend to Java/Spring Boot while maintaining 100% API compatibility with RuneLite plugins.
+
+**Technology Stack**:
+- Java 17
+- Spring Boot 3.2.0
+- MariaDB
+- Gradle
+- Flyway (database migrations)
+- Bouncy Castle (Blake2 hashing)
+
+---
+
+## ✅ Completed Components
+
+### 1. Project Setup
+- [x] Gradle build configuration (`build.gradle`)
+- [x] Application configuration (`application.yml`)
+- [x] Main application class (`GroupIronmenApplication.java`)
+- [x] Package structure created
+
+### 2. Database Layer
+- [x] Flyway migration script (`V1__init_schema.sql`)
+  - MariaDB-compatible schema
+  - JSON columns for arrays (instead of PostgreSQL arrays)
+  - All tables: groups, members, skills_*, collection_*
+  - Indexes and foreign keys
+
+### 3. Security & Authentication
+- [x] Blake2TokenHasher (`Blake2TokenHasher.java`)
+  - 100% compatible with Rust implementation
+  - 2-iteration Blake2b-256 hashing
+  - Token verification logic
+
+- [x] TokenAuthenticationFilter (`TokenAuthenticationFilter.java`)
+  - Extracts group_name from path
+  - Validates Authorization header
+  - Queries database for group_id
+  - Sets Spring Security context
+
+- [x] SecurityConfig (`SecurityConfig.java`)
+  - Public endpoints (no auth): /api/create-group, /api/ge-prices, etc.
+  - Protected endpoints: /api/group/**
+  - Stateless session management
+
+- [x] CorsConfig (`CorsConfig.java`)
+  - Configurable allowed origins
+  - Supports RuneLite plugin + frontend
+
+---
+
+## 🚧 Next Steps (In Order of Priority)
+
+### Phase 1A: Core Data Layer (CRITICAL)
+
+#### 1. JPA Entities
+**Files to create**:
+- `model/Group.java`
+- `model/Member.java`
+- `model/SkillDataDay.java`, `SkillDataMonth.java`, `SkillDataYear.java`
+- `model/CollectionTab.java`, `CollectionPage.java`
+- `model/CollectionLog.java`, `CollectionLogNew.java`
+
+**Key Requirements**:
+- Use `@Type(JsonType.class)` for array fields (Hibernate JSON support)
+- Map to MariaDB schema exactly
+- Include all timestamp fields (*_last_update)
+- Lombok annotations (@Data, @Entity, @Table)
+
+#### 2. Spring Data Repositories
+**Files to create**:
+- `repository/GroupRepository.java`
+- `repository/MemberRepository.java`
+- `repository/SkillDataRepository.java`
+- `repository/CollectionLogRepository.java`
+
+**Key Methods**:
+```java
+// GroupRepository
+Optional<Long> findGroupIdByNameAndTokenHash(String name, String hash);
+Optional<Group> findByGroupName(String name);
+
+// MemberRepository
+List<Member> findByGroupIdAndLastUpdatedAfter(Long groupId, Instant timestamp);
+Optional<Member> findByGroupIdAndMemberName(Long groupId, String name);
+int countByGroupId(Long groupId);
+```
+
+### Phase 1B: Service Layer
+
+#### 3. Business Logic Services
+**Files to create**:
+- `service/GroupService.java`
+  - `createGroup(CreateGroupRequest)` → Returns token + groupId
+  - `getGroupData(groupId, fromTimestamp)` → Delta updates
+
+- `service/MemberService.java`
+  - `updateMember(groupId, memberName, updateData)`
+  - `addMember(groupId, memberName)`
+  - `deleteMember(groupId, memberName)`
+  - `renameMember(groupId, oldName, newName)`
+
+- `service/SkillAggregationService.java`
+  - `aggregateSkills(period)` → Scheduled task
+  - `applyRetentionPolicy(period, maxAge)`
+
+- `service/GrandExchangeService.java`
+  - `fetchAndCachePrices()` → HTTP call to RuneScape Wiki API
+  - `getCachedPrices()` → Return cached prices
+
+- `service/CollectionLogService.java`
+  - `updateCollectionLog(memberId, collectionLogData)`
+  - `getCollectionLog(groupId)`
+
+### Phase 1C: API Controllers
+
+#### 4. REST Controllers
+**Files to create**:
+- `controller/PublicController.java`
+  ```java
+  POST /api/create-group
+  GET /api/ge-prices
+  GET /api/captcha-enabled
+  GET /api/collection-log-info
+  ```
+
+- `controller/GroupController.java`
+  ```java
+  GET /api/group/{group_name}/get-group-data?from_time=<timestamp>
+  GET /api/group/{group_name}/am-i-logged-in
+  GET /api/group/{group_name}/am-i-in-group
+  ```
+
+- `controller/MemberController.java`
+  ```java
+  POST /api/group/{group_name}/update-group-member
+  POST /api/group/{group_name}/add-group-member
+  DELETE /api/group/{group_name}/delete-group-member
+  PUT /api/group/{group_name}/rename-group-member
+  ```
+
+- `controller/SkillController.java`
+  ```java
+  GET /api/group/{group_name}/get-skill-data?period=<day|month|year>
+  ```
+
+- `controller/CollectionLogController.java`
+  ```java
+  GET /api/group/{group_name}/collection-log
+  ```
+
+#### 5. DTOs (Data Transfer Objects)
+**Files to create**:
+- `dto/CreateGroupRequest.java`
+- `dto/CreateGroupResponse.java`
+- `dto/UpdateMemberRequest.java`
+- `dto/GroupMemberResponse.java`
+- `dto/SkillDataResponse.java`
+- `dto/GePricesResponse.java`
+- `dto/CollectionLogResponse.java`
+
+**Critical**: DTOs must match Rust JSON structure EXACTLY for plugin compatibility.
+
+### Phase 1D: Background Jobs
+
+#### 6. Scheduled Tasks
+**File to create**:
+- `service/ScheduledTasks.java`
+  ```java
+  @Scheduled(fixedRate = 14400000) // 4 hours
+  public void updateGePrices()
+
+  @Scheduled(fixedRate = 1800000) // 30 minutes
+  public void aggregateSkills()
+  ```
+
+### Phase 1E: Exception Handling
+
+#### 7. Custom Exceptions & Global Handler
+**Files to create**:
+- `exception/GroupNotFoundException.java`
+- `exception/MemberNotFoundException.java`
+- `exception/GroupFullException.java` (max 5 members)
+- `exception/ValidationException.java`
+- `exception/GlobalExceptionHandler.java` (@ControllerAdvice)
+
+### Phase 1F: Utilities
+
+#### 8. Helper Classes
+**Files to create**:
+- `util/ValidationUtils.java`
+  - `validateMemberName(name)` → Regex: `[A-Za-z 0-9-_]{1,16}`
+  - `validateArrayLengths(member)` → Ensure correct array sizes
+
+- `util/CollectionLogLoader.java`
+  - Load `collection_log_info.json` at startup
+  - Populate `collection_page` table
+
+---
+
+## 🧪 Phase 2: Testing
+
+### Unit Tests
+- [ ] Blake2TokenHasher test (verify matches Rust output)
+- [ ] Service layer tests (Mockito)
+- [ ] Repository tests (TestContainers + MariaDB)
+
+### Integration Tests
+- [ ] Full API flow tests (create group → update member → get data)
+- [ ] Authentication tests (valid/invalid tokens)
+- [ ] CORS tests
+
+### Plugin Compatibility Tests
+- [ ] Test with actual RuneLite plugin
+- [ ] Verify JSON payload structure matches
+- [ ] Test all plugin → server endpoints
+
+---
+
+## 📦 Phase 3: Deployment
+
+### Docker Configuration
+**Files to create**:
+- `Dockerfile` (multi-stage build)
+- `docker-compose.yml` (backend + MariaDB)
+- `.dockerignore`
+- Deployment scripts for Proxmox/Debian containers
+
+### Environment Variables
+**Required for production**:
+```
+DB_HOST=<mariadb_host>
+DB_PORT=3306
+DB_NAME=groupironman
+DB_USER=<user>
+DB_PASSWORD=<password>
+SERVER_PORT=8080
+BACKEND_SECRET=<generate_strong_secret>
+CAPTCHA_ENABLED=false
+CORS_ORIGINS=https://yourfrontend.com
+```
+
+---
+
+## 📋 Implementation Checklist
+
+### Critical Path (Must Complete for MVP)
+- [ ] JPA Entities (Group, Member, SkillData, CollectionLog)
+- [ ] Repositories (GroupRepository, MemberRepository)
+- [ ] GroupService (createGroup, getGroupData)
+- [ ] MemberService (updateMember, addMember)
+- [ ] PublicController (createGroup, gePrices)
+- [ ] GroupController (getGroupData)
+- [ ] MemberController (updateMember)
+- [ ] DTOs matching Rust JSON structure
+- [ ] GrandExchangeService (fetch prices from Wiki API)
+- [ ] ScheduledTasks (GE prices updater)
+- [ ] Exception handling (@ControllerAdvice)
+- [ ] Docker build & deployment config
+
+### Nice-to-Have (Post-MVP)
+- [ ] Skill aggregation service
+- [ ] Collection log features
+- [ ] Captcha validation
+- [ ] Metrics/monitoring (Spring Actuator)
+- [ ] Comprehensive test suite
+
+---
+
+## 🐛 Known Challenges
+
+### 1. Array Type Handling
+**Issue**: MariaDB doesn't natively support arrays like PostgreSQL.
+**Solution**: Use JSON columns and custom Hibernate type converters.
+
+**Example**:
+```java
+@Type(JsonType.class)
+@Column(name = "skills", columnDefinition = "json")
+private List<Integer> skills; // 24 skills
+```
+
+### 2. Timestamp Precision
+**Issue**: PostgreSQL TIMESTAMPTZ vs MariaDB TIMESTAMP
+**Solution**: Use `TIMESTAMP(6)` for microsecond precision, convert to/from `Instant` in Java.
+
+### 3. Blake2 Hashing Compatibility
+**Critical**: Hash output MUST match Rust implementation exactly.
+**Testing**: Create unit test with known token/salt/hash triplets from Rust server.
+
+### 4. JSON Payload Structure
+**Critical**: Plugin expects specific JSON keys and types.
+**Solution**: Create DTOs that serialize EXACTLY as Rust structs.
+
+**Example from Rust**:
+```rust
+GroupMember {
+    name: String,
+    stats: Option<Vec<i32>>,  // null if not updated
+    skills: Option<Vec<i32>>,
+    // ...
+}
+```
+
+**Java DTO must output**:
+```json
+{
+  "name": "PlayerName",
+  "stats": [99, 99, 100, 301],  // or null
+  "skills": [13034431, ...],    // or null
+  "last_updated": "2024-01-01T12:00:00Z"
+}
+```
+
+---
+
+## 📞 Next Actions
+
+1. **Immediate**: Create JPA entities and repositories
+2. **Then**: Implement GroupService and MemberService
+3. **Then**: Create REST controllers with DTOs
+4. **Test**: Run Gradle build, start application, test endpoints with Postman
+5. **Deploy**: Create Docker image, test in container
+6. **Plugin Test**: Point RuneLite plugin to new backend, verify functionality
+
+---
+
+## 🔗 Related Documentation
+
+- Rust Source: `group-ironmen-master/server/src/`
+- Database Schema: `group-ironmen-master/server/src/db.rs` (lines 959-1170)
+- API Endpoints: `group-ironmen-master/server/src/authed.rs` + `unauthed.rs`
+- Frontend Site: `group-ironmen-master/site/src/`
+- Collection Log Info: `group-ironmen-master/site/public/data/collection_log_info.json`
+
+---
+
+**Status**: Phase 1A in progress (foundation complete, data layer next)
+**Last Updated**: 2025-10-27