Commit Graph

17 Commits

Author SHA1 Message Date
WquGuru 295124c1fa test(trader): add comprehensive unit tests and CI coverage reporting (#823)
* chore(config): add Python and uv support to project
- Add comprehensive Python .gitignore rules (pycache, venv, pytest, etc.)
- Add uv package manager specific ignores (.uv/, uv.lock)
- Initialize pyproject.toml for Python tooling
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* chore(deps): add testing dependencies
- Add github.com/stretchr/testify v1.11.1 for test assertions
- Add github.com/agiledragon/gomonkey/v2 v2.13.0 for mocking
- Promote github.com/rs/zerolog to direct dependency
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* ci(workflow): add PR test coverage reporting
Add GitHub Actions workflow to run unit tests and report coverage on PRs:
- Run Go tests with race detection and coverage profiling
- Calculate coverage statistics and generate detailed reports
- Post coverage results as PR comments with visual indicators
- Fix Go version to 1.23 (was incorrectly set to 1.25.0)
Coverage guidelines:
- Green (>=80%): excellent
- Yellow (>=60%): good
- Orange (>=40%): fair
- Red (<40%): needs improvement
This workflow is advisory only and does not block PR merging.
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* test(trader): add comprehensive unit tests for trader modules
Add unit test suites for multiple trader implementations:
- aster_trader_test.go: AsterTrader functionality tests
- auto_trader_test.go: AutoTrader lifecycle and operations tests
- binance_futures_test.go: Binance futures trader tests
- hyperliquid_trader_test.go: Hyperliquid trader tests
- trader_test_suite.go: Common test suite utilities and helpers
Also fix minor formatting issue in auto_trader.go (trailing whitespace)
Co-authored-by: tinkle-community <tinklefund@gmail.com>
* test(trader): preserve existing calculatePnLPercentage unit tests
Merge existing calculatePnLPercentage tests with incoming comprehensive test suite:
- Preserve TestCalculatePnLPercentage with 9 test cases covering edge cases
- Preserve TestCalculatePnLPercentage_RealWorldScenarios with 3 trading scenarios
- Add math package import for floating-point precision comparison
- All tests validate PnL percentage calculation with different leverage scenarios
Co-authored-by: tinkle-community <tinklefund@gmail.com>
---------
Co-authored-by: tinkle-community <tinklefund@gmail.com>
2025-11-09 17:43:28 +08:00
Lawrence Liu 8107667796 fix(database): prevent data loss on Docker restart with WAL mode and graceful shutdown (#817)
* fix(database): prevent data loss on Docker restart with WAL mode and graceful shutdown
Fixes #816
## Problem
Exchange API keys and private keys were being lost after `docker compose restart`.
This P0 bug posed critical security and operational risks.
### Root Cause
1. **SQLite journal_mode=delete**: Traditional rollback journal doesn't protect
   against data loss during non-graceful shutdowns
2. **Incomplete graceful shutdown**: Application relied on `defer database.Close()`
   which may not execute before process termination
3. **Docker grace period**: Default 10s may not be sufficient for cleanup
### Data Loss Scenario
```
User updates exchange config → Backend writes to SQLite → Data in buffer (not fsynced)
→ Docker restart (SIGTERM) → App exits → SQLite never flushes → Data lost
```
## Solution
### 1. Enable WAL Mode (Primary Fix)
- **Before**: `journal_mode=delete` (rollback journal)
- **After**: `journal_mode=WAL` (Write-Ahead Logging)
**Benefits:**
-  Crash-safe even during power loss
-  Better concurrent write performance
-  Atomic commits with durability guarantees
### 2. Improve Graceful Shutdown
**Before:**
```go
<-sigChan
traderManager.StopAll()
// defer database.Close() may not execute in time
```
**After:**
```go
<-sigChan
traderManager.StopAll()    // Step 1: Stop traders
server.Shutdown()          // Step 2: Stop HTTP server (new)
database.Close()           // Step 3: Explicit database close (new)
```
### 3. Increase Docker Grace Period
```yaml
stop_grace_period: 30s  # Allow 30s for graceful shutdown
```
## Changes
### config/database.go
- Enable `PRAGMA journal_mode=WAL` on database initialization
- Set `PRAGMA synchronous=FULL` for data durability
- Add log message confirming WAL mode activation
### api/server.go
- Add `httpServer *http.Server` field to Server struct
- Implement `Shutdown()` method with 5s timeout
- Replace `router.Run()` with `httpServer.ListenAndServe()` for graceful shutdown support
- Add `context` import for shutdown context
### main.go
- Add explicit shutdown sequence:
  1. Stop all traders
  2. Shutdown HTTP server (new)
  3. Close database connection (new)
- Add detailed logging for each shutdown step
### docker-compose.yml
- Add `stop_grace_period: 30s` to backend service
### config/database_test.go (TDD)
- `TestWALModeEnabled`: Verify WAL mode is active
- `TestSynchronousMode`: Verify synchronous=FULL setting
- `TestDataPersistenceAcrossReopen`: Simulate Docker restart scenario
- `TestConcurrentWritesWithWAL`: Verify concurrent write handling
## Test Results
```bash
$ go test -v ./config
=== RUN   TestWALModeEnabled
--- PASS: TestWALModeEnabled (0.25s)
=== RUN   TestSynchronousMode
--- PASS: TestSynchronousMode (0.06s)
=== RUN   TestDataPersistenceAcrossReopen
--- PASS: TestDataPersistenceAcrossReopen (0.05s)
=== RUN   TestConcurrentWritesWithWAL
--- PASS: TestConcurrentWritesWithWAL (0.09s)
PASS
```
All 16 tests pass (including 9 existing + 4 new WAL tests + 3 concurrent tests).
## Impact
**Before:**
- 🔴 Exchange credentials lost on restart
- 🔴 Trading operations disrupted
- 🔴 Security risk from credential re-entry
**After:**
-  Data persistence guaranteed
-  No credential loss after restart
-  Safe graceful shutdown in all scenarios
-  Better concurrent performance
## Acceptance Criteria
- [x] WAL mode enabled in database initialization
- [x] Graceful shutdown explicitly closes database
- [x] Unit tests verify data persistence across restarts
- [x] Docker grace period increased to 30s
- [x] All tests pass
## Deployment Notes
After deploying this fix:
1. Rebuild Docker image: `./start.sh start --build`
2. Existing `config.db` will be automatically converted to WAL mode
3. WAL files (`config.db-wal`, `config.db-shm`) will be created
4. No manual intervention required
## References
- SQLite WAL Mode: https://www.sqlite.org/wal.html
- Go http.Server Graceful Shutdown: https://pkg.go.dev/net/http#Server.Shutdown
* Add config.db* to gitignore
2025-11-09 16:23:00 +08:00
tinkle-community f7af75c657 update .gitignore 2025-11-08 17:25:00 +08:00
Icyoung 89085173f9 Dev Crypto (#730)
* feat: remove admin mode
* feat: bugfix
* feat(crypto): 添加RSA-OAEP + AES-GCM混合加密服务
- 实现CryptoService加密服务,支持RSA-OAEP-2048 + AES-256-GCM混合加密
- 集成数据库层加密,自动加密存储敏感字段(API密钥、私钥等)
- 支持环境变量DATA_ENCRYPTION_KEY配置数据加密密钥
- 适配SQLite数据库加密存储(从PostgreSQL移植)
- 保持Hyperliquid代理钱包处理兼容性
- 更新.gitignore以正确处理crypto模块代码
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
* feat(scripts): 添加加密环境一键设置脚本
- setup_encryption.sh: 一键生成RSA密钥对+数据加密密钥+JWT密钥
- generate_rsa_keys.sh: 专业的RSA-2048密钥对生成工具
- generate_data_key.sh: 生成AES-256数据加密密钥和JWT认证密钥
- ENCRYPTION_README.md: 详细的加密系统说明文档
- 支持自动检测现有密钥并只生成缺失的密钥
- 完善的权限管理和安全验证
- 兼容macOS和Linux的跨平台支持
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
* feat(api): 添加加密API端点和Gin框架集成
- 新增CryptoHandler处理加密相关API请求
- 提供/api/crypto/public-key端点获取RSA公钥
- 提供/api/crypto/decrypt端点解密敏感数据
- 适配Gin框架的HTTP处理器格式
- 集成CryptoService到API服务器
- 支持前端加密数据传输和解密
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
* feat(web): 添加前端加密服务和两阶段密钥输入组件
- CryptoService: Web Crypto API集成,支持RSA-OAEP加密
- TwoStageKeyModal: 安全的两阶段私钥输入组件,支持剪贴板混淆
- 完善国际化翻译支持加密相关UI文本
- 修复TypeScript类型错误和编译问题
- 支持前端敏感数据加密传输到后端
- 增强用户隐私保护和数据安全
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
* feat(auth): 增强JWT认证安全性
- 优先使用环境变量JWT_SECRET而不是数据库配置
- 支持通过.env文件安全配置JWT认证密钥
- 保留数据库配置作为回退机制
- 改进JWT密钥来源日志显示
- 增强系统启动时的安全配置检查
- 支持运行时动态JWT密钥切换
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
* feat(docker): 集成加密环境变量到Docker部署
- 添加DATA_ENCRYPTION_KEY环境变量传递到容器
- 添加JWT_SECRET环境变量支持
- 挂载secrets目录使容器可访问RSA密钥文件
- 确保容器内加密服务正常工作
- 解决容器启动失败和加密初始化问题
- 完善Docker Compose加密环境配置
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
* feat(start): 集成自动加密环境检测和设置
- 增强check_encryption()函数检测JWT_SECRET和DATA_ENCRYPTION_KEY
- 自动运行setup_encryption.sh当检测到缺失密钥时
- 改进加密状态显示,包含RSA+AES+JWT全套加密信息
- 优化用户体验,提供清晰的加密配置反馈
- 支持一键设置完整加密环境
- 确保容器启动前加密环境就绪
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
* feat: format fix
* fix(security): 修复前端模型和交易所配置敏感数据明文传输
- 在handleSaveModelConfig中对API密钥进行RSA-OAEP加密
- 在handleSaveExchangeConfig中对API密钥、Secret密钥和Aster私钥进行加密
- 只有非空敏感数据才进行加密处理
- 添加加密失败错误处理和用户友好提示
- 增加encryptionFailed翻译键的中英文支持
- 使用用户ID和会话ID作为加密上下文增强安全性
这修复了之前敏感数据在网络传输中以明文形式发送的安全漏洞。
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
* fix(crypto): 修复后端加密服务集成和缺失的加密端点
- 添加Server结构体缺少的cryptoService字段
- 实现handleUpdateModelConfigsEncrypted处理器用于模型配置加密传输
- 修复handleUpdateExchangeConfigsEncrypted中的函数调用
- 在前端API中添加updateModelConfigsEncrypted方法
- 统一RSA密钥路径从secrets/rsa_key改为keys/rsa_private.key
- 确保前端可以使用加密端点安全传输敏感数据
- 兼容原有加密通信模式和二段输入私钥功能
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
---------
Co-authored-by: icy <icyoung520@gmail.com>
Co-authored-by: tinkle-community <tinklefund@gmail.com>
2025-11-08 02:03:09 +08:00
SkywalkerJi 1c4c933b7f refactor(decision): use XML tags to separate reasoning from JSON decisions (#719)
* Separate the AI's thought process from the instruction JSON using XML tags.
* Avoid committing encryption key related materials to Git.
* Removing adaptive series prompts, awaiting subsequent modifications for compatibility.
2025-11-07 22:35:53 +08:00
Burt 900323b386 Fix: 提示词, 竞赛数据接口在管理员模式下转为公开 (#607)
* 提示词, 竞赛数据接口在管理员模式下转为公开
* Fix "go vet" error
2025-11-06 20:42:43 +08:00
Ember b79878ab36 feat: add ESLint and Prettier with pre-commit hook
- Install ESLint 9 with TypeScript and React support
- Install Prettier with custom configuration (no semicolons)
- Add husky and lint-staged for pre-commit hooks
- Configure lint-staged to auto-fix and format on commit
- Relax ESLint rules to avoid large-scale code changes
- Format all existing code with Prettier (no semicolons)
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
2025-11-05 11:41:14 +08:00
zbhan fcacfbd79d Feat: Update docs
- 重构文档结构
- 更新文档内容
- 制定roadmap
- 提供中/EN 双语文档
2025-11-01 15:05:24 -04:00
SkywalkerJi c3e37f83df Ignore the database to avoid submitting sensitive data. 2025-10-31 18:30:36 +08:00
d0lwl0b 0c599ba70b chore: add environment variable support and enhance startup script
- Add .env.example template for configurable Docker deployment settings
- Update docker-compose.yml to support environment-driven port/timezone configuration
- Refactor start.sh with improved Docker Compose detection, environment validation, and automated frontend building
- Enhance script documentation and error handling for better maintainability
2025-10-29 22:35:01 +08:00
awake 0x 9771fbdebf sync: Sync complete codebase from nofx internal version
Full code synchronization from nofx internal repository to open-nofx.
Updated all source files, documentation, and configuration.
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
2025-10-29 12:00:09 +08:00
tinkle-community 6b3959b611 sync: Sync codebase from nofx internal version
- Update .gitignore to align with nofx configuration
- Sync README files across all languages (EN/ZH/RU/UK)
- Update web/package-lock.json with latest dependencies
This commit synchronizes the open-nofx repository with the latest
changes from the internal nofx version to maintain consistency.
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
2025-10-29 11:15:08 +08:00
tinkle-community f3720699eb chore: Remove unnecessary files from open-nofx repository
Remove node_modules directory and update .gitignore:
- Remove all web/node_modules files from git tracking
- Add node_modules/, web/dist/, web/.vite/ to .gitignore
- Remove backup files (*.bak) and empty directories (web/web/)
- Align open-nofx repository structure with nofx internal version
This cleanup ensures the open-source version only contains necessary
source files and proper gitignore configuration.
Co-Authored-By: tinkle-community <tinklefund@gmail.com>
2025-10-29 10:44:27 +08:00
tinkle-community f940196f34 chore: Remove nofx_test binary and add to gitignore 2025-10-29 06:15:27 +08:00
tinkle-community 8a26b8161b Feature: Add multi-language support and UI improvements
- Add language context and translation system (Chinese/English)
- Enhance UI components with i18n support
- Update AILearning, EquityChart, and CompetitionPage
- Add language toggle in header
- Improve user experience with localized text
2025-10-28 22:25:36 +08:00
tinkle-community a726702302 Update: Merge nofx improvements
- Frontend trading records and UI enhancements
- Optimized AI prompts and decision engine
- Performance analysis and comparison features
- Binance-style UI improvements
2025-10-28 21:45:28 +08:00
tinkle-community 5aa50d35d7 Initial commit: NOFX AI Trading System
- Multi-AI competition mode (Qwen vs DeepSeek)
- Binance Futures integration
- AI self-learning mechanism
- Professional web dashboard
- Complete risk management system
2025-10-28 15:47:34 +08:00