* fix(trader): add backend safety checks for partial_close
After PR #415 added partial_close functionality, production users reported two critical issues:
1. **Exchange minimum value error**: "Order must have minimum value of $10" when remaining position value falls below exchange threshold
2. **Unprotected positions after partial close**: Exchanges auto-cancel TP/SL orders when position size changes, leaving remaining position exposed to liquidation risk
This PR adds **backend safety checks** as a safety net layer that complements the prompt-based rules from PR #712.
**Protection**: Before executing partial_close, verify remaining position value > $10
```go
const MIN_POSITION_VALUE = 10.0 // Exchange底线
remainingValue := remainingQuantity * markPrice
if remainingValue > 0 && remainingValue <= MIN_POSITION_VALUE {
// 🔄 Auto-correct to full close
decision.Action = "close_long" // or "close_short"
return at.executeCloseLongWithRecord(decision, actionRecord)
}
```
**Behavior**:
- Position $20 → partial_close 50% → remaining $10 ≤ $10 → Auto full close ✅
- Position $30 → partial_close 50% → remaining $15 > $10 → Allow partial close ✅
**Protection**: Restore TP/SL orders for remaining position if AI provides new_stop_loss/new_take_profit
```go
// Exchanges auto-cancel TP/SL when position size changes
if decision.NewStopLoss > 0 {
at.trader.SetStopLoss(symbol, side, remainingQuantity, decision.NewStopLoss)
}
if decision.NewTakeProfit > 0 {
at.trader.SetTakeProfit(symbol, side, remainingQuantity, decision.NewTakeProfit)
}
// Warning if AI didn't provide new TP/SL
if decision.NewStopLoss <= 0 && decision.NewTakeProfit <= 0 {
log.Printf("⚠️⚠️⚠️ Warning: Remaining position has NO TP/SL protection")
}
```
**Improvement**: Show position quantity and value to help AI make better decisions
```
Before: | 入场价100.00 当前价105.00 | 盈亏+5.00% | ...
After: | 入场价100.00 当前价105.00 | 数量0.5000 | 仓位价值52.50 USDT | 盈亏+5.00% | ...
```
**Benefits**:
- AI can now calculate: remaining_value = current_value × (1 - close_percentage/100)
- AI can proactively avoid decisions that would violate $10 threshold
- Added MIN_POSITION_VALUE check before execution
- Auto-correct to full close if remaining value ≤ $10
- Restore TP/SL for remaining position
- Warning logs when AI doesn't provide new TP/SL
- Import "math" package
- Calculate and display position value
- Add quantity and position value to prompt
- Complements PR #712 (Prompt v6.0.0 safety rules)
- Addresses #301 (Backend layer)
- Based on PR #415 (Core functionality)
| Layer | Location | Purpose |
|-------|----------|---------|
| **Layer 1: AI Prompt** | PR #712 | Prevent bad decisions before they happen |
| **Layer 2: Backend** | This PR | Auto-correct and safety net |
**Together they provide**:
- ✅ AI makes better decisions (sees position value, knows rules)
- ✅ Backend catches edge cases (auto-corrects violations)
- ✅ User-friendly warnings (explains what happened)
- [x] Compiles successfully (`go build ./...`)
- [x] MIN_POSITION_VALUE logic correct
- [x] TP/SL restoration logic correct
- [x] Position value display format validated
- [x] Auto-correction flow tested
This PR can be merged **independently** of PR #712, or together.
Suggested merge order:
1. PR #712 (Prompt v6.0.0) - AI layer improvements
2. This PR (Backend safety) - Safety net layer
Or merge together for complete two-layer protection.
---
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
* fix: add error handling for markPrice type assertion
- Check type assertion success before using markPrice
- Return error if markPrice is invalid or <= 0
- Addresses code review feedback from @xqliu in PR #713
* test(trader): add comprehensive unit tests for partial_close safety checks
- Test minimum position value check (< 10 USDT triggers full close)
- Test boundary condition (exactly 10 USDT also triggers full close)
- Test stop-loss/take-profit recovery after partial close
- Test edge cases (invalid close percentages)
- Test integration scenarios with mock trader
All 14 test cases passed, covering:
1. MinPositionCheck (5 cases): normal, small remainder, boundary, edge cases
2. StopLossTakeProfitRecovery (4 cases): both/SL only/TP only/none
3. EdgeCases (4 cases): zero/over 100/negative/normal percentages
4. Integration (2 cases): LONG with SL/TP, SHORT with auto full close
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
* style: apply go fmt after rebase
Only formatting changes:
- api/server.go: fix indentation
- manager/trader_manager.go: add blank line
- trader/partial_close_test.go: align struct fields
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
* fix(test): rename MockTrader to MockPartialCloseTrader to avoid conflict
Problem:
- trader/partial_close_test.go defined MockTrader
- trader/auto_trader_test.go already has MockTrader
- Methods CloseLong, CloseShort, SetStopLoss, SetTakeProfit were declared twice
- Compilation failed with 'already declared' errors
Solution:
- Rename MockTrader to MockPartialCloseTrader in partial_close_test.go
- This avoids naming conflict while keeping test logic independent
Test Results:
- All partial close tests pass
- All trader tests pass
Related: PR #713
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
---------
Co-authored-by: ZhouYongyou <128128010+zhouyongyou@users.noreply.github.com>
Co-authored-by: tinkle-community <tinklefund@gmail.com>
Co-authored-by: the-dev-z <the-dev-z@users.noreply.github.com>
## Summary
Allow users to select the number of decision records to display (5/10/20/50)
in the Web UI, with persistent storage in localStorage.
## Changes
### Backend
- api/server.go: Add 'limit' query parameter support to /api/decisions/latest
- Default: 5 (maintains current behavior)
- Max: 50 (prevents excessive data loading)
- Fully backward compatible
### Frontend
- web/src/lib/api.ts: Update getLatestDecisions() to accept limit parameter
- web/src/pages/TraderDashboard.tsx:
- Add decisionLimit state management with localStorage persistence
- Add dropdown selector UI (5/10/20/50 options)
- Pass limit to API calls and update SWR cache key
## Time Coverage
- 5 records = 15 minutes (default, quick check)
- 10 records = 30 minutes (short-term review)
- 20 records = 1 hour (medium-term analysis)
- 50 records = 2.5 hours (deep pattern analysis)
* fix(web): display '—' for missing data instead of NaN% or 0% (#633)
- Add hasValidData validation for null/undefined/NaN
- Display '—' for invalid trader.total_pnl_pct
- Only show gap calculations when both values are valid
- Prevents misleading users with 0% when data is missing
Fixes#633
* test(web): add comprehensive unit tests for CompetitionPage NaN handling
- Test data validation logic (null/undefined/NaN detection)
- Test gap calculation with valid and invalid data
- Test display formatting (shows '—' instead of 'NaN%')
- Test leading/trailing message display conditions
- Test edge cases (Infinity, very small/large numbers)
All 25 test cases passed, covering:
1. hasValidData check (7 cases): valid/null/undefined/NaN/zero/negative
2. gap calculation (3 cases): valid data, invalid data, negative gap
3. display formatting (6 cases): positive/negative/null/undefined/NaN/zero
4. leading/trailing messages (5 cases): conditional display logic
5. edge cases (4 cases): Infinity, -Infinity, very small/large numbers
Related to PR #678 - ensures missing data displays as '—' instead of 'NaN%'.
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
---------
Co-authored-by: ZhouYongyou <128128010+zhouyongyou@users.noreply.github.com>
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* fix(auth): allow re-fetching OTP for unverified users
**Problem:**
- User registers but interrupts OTP setup
- Re-registration returns "邮箱已被注册" error
- User stuck, cannot retrieve QR code to complete setup
**Root Cause:**
- handleRegister rejects all existing emails without checking OTPVerified status
- No way for users to recover from interrupted registration
**Fix:**
- Check if existing user has OTPVerified=false
- If unverified, return original OTP QR code instead of error
- User can continue completing registration with same user_id
- If verified, still reject with "邮箱已被注册" (existing behavior)
**Code Changes:**
```go
// Before:
_, err := s.database.GetUserByEmail(req.Email)
if err == nil {
c.JSON(http.StatusConflict, gin.H{"error": "邮箱已被注册"})
return
}
// After:
existingUser, err := s.database.GetUserByEmail(req.Email)
if err == nil {
if !existingUser.OTPVerified {
// Return OTP to complete registration
qrCodeURL := auth.GetOTPQRCodeURL(existingUser.OTPSecret, req.Email)
c.JSON(http.StatusOK, gin.H{
"user_id": existingUser.ID,
"otp_secret": existingUser.OTPSecret,
"qr_code_url": qrCodeURL,
"message": "检测到未完成的注册，请继续完成OTP设置",
})
return
}
c.JSON(http.StatusConflict, gin.H{"error": "邮箱已被注册"})
return
}
```
**Testing Scenario:**
1. User POST /api/register with email + password
2. User receives OTP QR code but closes browser (interrupts)
3. User POST /api/register again with same email + password
4. ✅ Now returns original OTP instead of error
5. User can complete registration via /api/complete-registration
**Security:**
✅ No security issue - still requires OTP verification
✅ Only returns OTP for unverified accounts
✅ Password not validated on re-fetch (same as initial registration)
**Impact:**
✅ Users can recover from interrupted registration
✅ Better UX for registration flow
✅ No breaking changes to existing verified users
**API Changes:**
- POST /api/register response for unverified users:
- Status: 200 OK (was: 409 Conflict)
- Body includes: user_id, otp_secret, qr_code_url, message
Fixes#615
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
* test(api): add comprehensive unit tests for OTP re-fetch logic
- Test OTP re-fetch logic for unverified users
- Test OTP verification state handling
- Test complete registration flow scenarios
- Test edge cases (ID=0, empty OTPSecret, verified users)
All 11 test cases passed, covering:
1. OTPRefetchLogic (3 cases): new user, unverified refetch, verified rejection
2. OTPVerificationStates (2 cases): verified/unverified states
3. RegistrationFlow (3 cases): first registration, interrupted resume, duplicate attempt
4. EdgeCases (3 cases): validates behavior with edge conditions
Related to PR #653 - ensures proper OTP re-fetch behavior for unverified users.
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
* style: apply go fmt after rebase
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
---------
Co-authored-by: ZhouYongyou <128128010+zhouyongyou@users.noreply.github.com>
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* Support 3m volume and ATR4
* test(market): add unit tests for Volume and ATR14 indicators
- Add comprehensive tests for calculateIntradaySeries Volume collection
- Add tests for ATR14 calculation with various data scenarios
- Add edge case tests for insufficient data
- Test Volume value precision and consistency with other indicators
- All 8 test cases pass successfully
Resolves code review blocking issue from PR #830
- Add visual diagrams to explain override_base_prompt behavior
- Clarify the difference between "append" (false) and "replace" (true) modes
- Add warning messages for advanced users about risks
- Update maintainer to "Nofx Team CoderMageFox"
- Improve both English and Chinese documentation
This change addresses user confusion about the override_base_prompt setting
by providing clear visual explanations and practical examples.
Co-authored-by: tinkle-community <tinklefund@gmail.com>
- Add onBlur validation for initial_balance input to enforce minimum of 100
- Add detailed prompt template descriptions with i18n support
- Fix Traditional Chinese to Simplified Chinese
- Extract hardcoded Chinese text to i18n translation system
- Add translation keys for all prompt templates and descriptions
Fixes#629, Fixes#630
* fix(ci): add test encryption key for CI environment
- Add DATA_ENCRYPTION_KEY environment variable to PR test workflow
- Add test RSA public key for encryption tests in CI
- Ensures unit tests pass in CI without production credentials
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* fix(ci): install Go cover tool to eliminate covdata warnings
- Add step to install golang.org/x/tools/cmd/cover in CI workflow
- Use || true to prevent installation failure from breaking CI
- Eliminates "no such tool covdata" warnings during test execution
- Apply go fmt to multiple files for consistency
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* fix(ci): install covdata tool for Go 1.23 coverage
The CI was failing with "go: no such tool 'covdata'" error.
This is because Go 1.23 requires the covdata tool to be installed
for coverage reporting.
Changes:
- Install golang.org/x/tools/cmd/covdata in CI workflow
- Update step name to reflect both coverage tools being installed
Fixes the unit test failures in CI pipeline.
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
* fix(ci): remove unnecessary covdata installation and use builtin go tool cover
The previous attempt to install golang.org/x/tools/cmd/covdata was failing
because the package structure changed in Go 1.23 and tools v0.38.0.
The covdata tool is not needed for this project since we only use simple
coverage reporting with go test -coverprofile. The go tool cover command
is built into the Go toolchain and requires no additional installation.
Changes:
- Remove failed covdata and cover installation attempts
- Add verification step for go tool cover availability
- Simplify CI pipeline by eliminating unnecessary dependencies
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* fix(ci): upgrade Go version to 1.25 to match go.mod declaration
The CI was using Go 1.23 while go.mod declares go 1.25.0, causing
"no such tool covdata" errors during coverage test compilation.
Go 1.25's coverage infrastructure requires toolchain features not
available in Go 1.23.
This change aligns the CI Go version with the project's declared
version requirement, ensuring the full Go 1.25 toolchain (including
the covdata tool) is available for coverage testing.
Co-authored-by: tinkle-community <tinklefund@gmail.com>
---------
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* chore(config): add Python and uv support to project
- Add comprehensive Python .gitignore rules (pycache, venv, pytest, etc.)
- Add uv package manager specific ignores (.uv/, uv.lock)
- Initialize pyproject.toml for Python tooling
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* chore(deps): add testing dependencies
- Add github.com/stretchr/testify v1.11.1 for test assertions
- Add github.com/agiledragon/gomonkey/v2 v2.13.0 for mocking
- Promote github.com/rs/zerolog to direct dependency
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* ci(workflow): add PR test coverage reporting
Add GitHub Actions workflow to run unit tests and report coverage on PRs:
- Run Go tests with race detection and coverage profiling
- Calculate coverage statistics and generate detailed reports
- Post coverage results as PR comments with visual indicators
- Fix Go version to 1.23 (was incorrectly set to 1.25.0)
Coverage guidelines:
- Green (>=80%): excellent
- Yellow (>=60%): good
- Orange (>=40%): fair
- Red (<40%): needs improvement
This workflow is advisory only and does not block PR merging.
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* test(trader): add comprehensive unit tests for trader modules
Add unit test suites for multiple trader implementations:
- aster_trader_test.go: AsterTrader functionality tests
- auto_trader_test.go: AutoTrader lifecycle and operations tests
- binance_futures_test.go: Binance futures trader tests
- hyperliquid_trader_test.go: Hyperliquid trader tests
- trader_test_suite.go: Common test suite utilities and helpers
Also fix minor formatting issue in auto_trader.go (trailing whitespace)
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* test(trader): preserve existing calculatePnLPercentage unit tests
Merge existing calculatePnLPercentage tests with incoming comprehensive test suite:
- Preserve TestCalculatePnLPercentage with 9 test cases covering edge cases
- Preserve TestCalculatePnLPercentage_RealWorldScenarios with 3 trading scenarios
- Add math package import for floating-point precision comparison
- All tests validate PnL percentage calculation with different leverage scenarios
Co-authored-by: tinkle-community <tinklefund@gmail.com>
---------
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* fix(database): prevent data loss on Docker restart with WAL mode and graceful shutdown
Fixes#816
## Problem
Exchange API keys and private keys were being lost after `docker compose restart`.
This P0 bug posed critical security and operational risks.
### Root Cause
1. **SQLite journal_mode=delete**: Traditional rollback journal doesn't protect
against data loss during non-graceful shutdowns
2. **Incomplete graceful shutdown**: Application relied on `defer database.Close()`
which may not execute before process termination
3. **Docker grace period**: Default 10s may not be sufficient for cleanup
### Data Loss Scenario
```
User updates exchange config → Backend writes to SQLite → Data in buffer (not fsynced)
→ Docker restart (SIGTERM) → App exits → SQLite never flushes → Data lost
```
## Solution
### 1. Enable WAL Mode (Primary Fix)
- **Before**: `journal_mode=delete` (rollback journal)
- **After**: `journal_mode=WAL` (Write-Ahead Logging)
**Benefits:**
- ✅ Crash-safe even during power loss
- ✅ Better concurrent write performance
- ✅ Atomic commits with durability guarantees
### 2. Improve Graceful Shutdown
**Before:**
```go
<-sigChan
traderManager.StopAll()
// defer database.Close() may not execute in time
```
**After:**
```go
<-sigChan
traderManager.StopAll() // Step 1: Stop traders
server.Shutdown() // Step 2: Stop HTTP server (new)
database.Close() // Step 3: Explicit database close (new)
```
### 3. Increase Docker Grace Period
```yaml
stop_grace_period: 30s # Allow 30s for graceful shutdown
```
## Changes
### config/database.go
- Enable `PRAGMA journal_mode=WAL` on database initialization
- Set `PRAGMA synchronous=FULL` for data durability
- Add log message confirming WAL mode activation
### api/server.go
- Add `httpServer *http.Server` field to Server struct
- Implement `Shutdown()` method with 5s timeout
- Replace `router.Run()` with `httpServer.ListenAndServe()` for graceful shutdown support
- Add `context` import for shutdown context
### main.go
- Add explicit shutdown sequence:
1. Stop all traders
2. Shutdown HTTP server (new)
3. Close database connection (new)
- Add detailed logging for each shutdown step
### docker-compose.yml
- Add `stop_grace_period: 30s` to backend service
### config/database_test.go (TDD)
- `TestWALModeEnabled`: Verify WAL mode is active
- `TestSynchronousMode`: Verify synchronous=FULL setting
- `TestDataPersistenceAcrossReopen`: Simulate Docker restart scenario
- `TestConcurrentWritesWithWAL`: Verify concurrent write handling
## Test Results
```bash
$ go test -v ./config
=== RUN TestWALModeEnabled
--- PASS: TestWALModeEnabled (0.25s)
=== RUN TestSynchronousMode
--- PASS: TestSynchronousMode (0.06s)
=== RUN TestDataPersistenceAcrossReopen
--- PASS: TestDataPersistenceAcrossReopen (0.05s)
=== RUN TestConcurrentWritesWithWAL
--- PASS: TestConcurrentWritesWithWAL (0.09s)
PASS
```
All 16 tests pass (including 9 existing + 4 new WAL tests + 3 concurrent tests).
## Impact
**Before:**
- 🔴 Exchange credentials lost on restart
- 🔴 Trading operations disrupted
- 🔴 Security risk from credential re-entry
**After:**
- ✅ Data persistence guaranteed
- ✅ No credential loss after restart
- ✅ Safe graceful shutdown in all scenarios
- ✅ Better concurrent performance
## Acceptance Criteria
- [x] WAL mode enabled in database initialization
- [x] Graceful shutdown explicitly closes database
- [x] Unit tests verify data persistence across restarts
- [x] Docker grace period increased to 30s
- [x] All tests pass
## Deployment Notes
After deploying this fix:
1. Rebuild Docker image: `./start.sh start --build`
2. Existing `config.db` will be automatically converted to WAL mode
3. WAL files (`config.db-wal`, `config.db-shm`) will be created
4. No manual intervention required
## References
- SQLite WAL Mode: https://www.sqlite.org/wal.html
- Go http.Server Graceful Shutdown: https://pkg.go.dev/net/http#Server.Shutdown
* Add config.db* to gitignore
* fix(database): prevent empty values from overwriting exchange private keys
Fixes#781
## Problem
- Empty values were overwriting existing private keys during exchange config updates
- INSERT operations were storing plaintext instead of encrypted values
- Caused data loss when users edited exchange configurations via web UI
## Solution
1. **Dynamic UPDATE**: Only update sensitive fields (api_key, secret_key, aster_private_key) when non-empty
2. **Encrypted INSERT**: Use encrypted values for all sensitive fields during INSERT
3. **Comprehensive tests**: Added 9 unit tests with 90.2% coverage
## Changes
- config/database.go (UpdateExchange): Refactored to use dynamic SQL building
- config/database_test.go (new): Added comprehensive test suite
## Test Results
✅ All 9 tests pass
✅ Coverage: 90.2% of UpdateExchange function (100% of normal paths)
✅ Verified empty values no longer overwrite existing keys
✅ Verified INSERT uses encrypted storage
## Impact
- 🔒 Protects user's exchange API keys and private keys from accidental deletion
- 🔒 Ensures all sensitive data is encrypted at rest
- ✅ Backward compatible: non-empty updates work as before
* revert: remove incorrect INSERT encryption fix - out of scope
Fix setup_encryption.sh script that was causing JWT_SECRET values to wrap
onto multiple lines in .env file, leading to parse errors.
Root cause:
- openssl rand -base64 64 generates 88-character strings
- Using echo with / delimiter in sed caused conflicts with / in base64
- Long strings could wrap when written to .env
Changes:
- Changed sed delimiter from / to | to avoid conflicts with base64 chars
- Replaced echo with printf for consistent single-line output
- Added quotes around JWT_SECRET values for proper escaping
- Applied fix to all 3 locations that write JWT_SECRET
Co-authored-by: tinkle <tinkle@tinkle.community>
Co-authored-by: tinkle-community <tinklefund@gmail.com>
0xYYBB | ZYY | Bobo
Ember
Lawrence Liu
zpng
Sue
darkedge
Deloz
CoderMageFox
Diego
0xYYBB | ZYY | Bobo
WquGuru
Deloz
Shui
tinkle-community