27 Commits

Author SHA1 Message Date
tinkle-community
495209a69b Refactor/trading actions (#1169)
* refactor: 简化交易动作,移除 update_stop_loss/update_take_profit/partial_close

- 移除 Decision 结构体中的 NewStopLoss, NewTakeProfit, ClosePercentage 字段
- 删除 executeUpdateStopLossWithRecord, executeUpdateTakeProfitWithRecord, executePartialCloseWithRecord 函数
- 简化 logger 中的 partial_close 聚合逻辑
- 更新 AI prompt 和验证逻辑,只保留 6 个核心动作
- 清理相关测试代码

保留的交易动作: open_long, open_short, close_long, close_short, hold, wait

* refactor: 移除 AI学习与反思 模块

- 删除前端 AILearning.tsx 组件和相关引用
- 删除后端 /performance API 接口
- 删除 logger 中 AnalyzePerformance、calculateSharpeRatio 等函数
- 删除 PerformanceAnalysis、TradeOutcome、SymbolPerformance 等结构体
- 删除 Context 中的 Performance 字段
- 移除 AI prompt 中夏普比率自我进化相关内容
- 清理 i18n 翻译文件中的相关条目

该模块基于磁盘存储计算,经常出错,做减法移除

* refactor: 将数据库操作统一迁移到 store 包

- 新增 store/ 包,统一管理所有数据库操作
  - store.go: 主 Store 结构,懒加载各子模块
  - user.go, ai_model.go, exchange.go, trader.go 等子模块
  - 支持加密/解密函数注入 (SetCryptoFuncs)

- 更新 main.go 使用 store.New() 替代 config.NewDatabase()
- 更新 api/server.go 使用 *store.Store 替代 *config.Database
- 更新 manager/trader_manager.go:
  - 新增 LoadTradersFromStore, LoadUserTradersFromStore 方法
  - 删除旧版 LoadUserTraders, LoadTraderByID, loadSingleTrader 等方法
  - 移除 nofx/config 依赖

- 删除 config/database.go 和 config/database_test.go
- 更新 api/server_test.go 使用 store.Trader 类型
- 清理 logger/ 包中未使用的 telegram 相关代码

* refactor: unify encryption key management via .env

- Remove redundant EncryptionManager and SecureStorage
- Simplify CryptoService to load keys from environment variables only
  - RSA_PRIVATE_KEY: RSA private key for client-server encryption
  - DATA_ENCRYPTION_KEY: AES-256 key for database encryption
  - JWT_SECRET: JWT signing key for authentication
- Update start.sh to auto-generate missing keys on first run
- Remove secrets/ directory and file-based key storage
- Delete obsolete encryption setup scripts
- Update .env.example with all required keys

* refactor: unify logger usage across mcp package

- Add MCPLogger adapter in logger package to implement mcp.Logger interface
- Update mcp/config.go to use global logger by default
- Remove redundant defaultLogger from mcp/logger.go
- Keep noopLogger for testing purposes

* chore: remove leftover test RSA key file

* chore: remove unused bootstrap package

* refactor: unify logging to use logger package instead of fmt/log

- Replace all fmt.Print/log.Print calls with logger package
- Add auto-initialization in logger package init() for test compatibility
- Update main.go to initialize logger at startup
- Migrate all packages: api, backtest, config, decision, manager, market, store, trader

* refactor: rename database file from config.db to data.db

- Update main.go, start.sh, docker-compose.yml
- Update migration script and documentation
- Update .gitignore and translations

* fix: add RSA_PRIVATE_KEY to docker-compose environment

* fix: add registration_enabled to /api/config response

* fix: Fix navigation between login and register pages

Use window.location.href instead of react-router's navigate() to fix
the issue where URL changes but the page doesn't reload due to App.tsx
using custom route state management.

* fix: Switch SQLite from WAL to DELETE mode for Docker compatibility

WAL mode causes data sync issues with Docker bind mounts on macOS due
to incompatible file locking mechanisms between the container and host.
DELETE mode (traditional journaling) ensures data is written directly
to the main database file.

* refactor: Remove default user from database initialization

The default user was a legacy placeholder that is no longer needed now
that proper user registration is in place.

* feat: Add order tracking system with centralized status sync

- Add trader_orders table for tracking all order lifecycle
- Implement GetOrderStatus interface for all exchanges (Binance, Bybit, Hyperliquid, Aster, Lighter)
- Create OrderSyncManager for centralized order status polling
- Add trading statistics (Sharpe ratio, win rate, profit factor) to AI context
- Include recent completed orders in AI decision input
- Remove per-order goroutine polling in favor of global sync manager

* feat: Add TradingView K-line chart to dashboard

- Create TradingViewChart component with exchange/symbol selectors
- Support Binance, Bybit, OKX, Coinbase, Kraken, KuCoin exchanges
- Add popular symbols quick selection
- Support multiple timeframes (1m to 1W)
- Add fullscreen mode
- Integrate with Dashboard page below equity chart
- Add i18n translations for zh/en

* refactor: Replace separate charts with tabbed ChartTabs component

- Create ChartTabs component with tab switching between equity curve and K-line
- Add embedded mode support for EquityChart and TradingViewChart
- User can now switch between account equity and market chart in same area

* fix: Use ChartTabs in App.tsx and fix embedded mode in EquityChart

- Replace EquityChart with ChartTabs in App.tsx (the actual dashboard renderer)
- Fix EquityChart embedded mode for error and empty data states
- Rename interval state to timeInterval to avoid shadowing window.setInterval
- Add debug logging to ChartTabs component

* feat: Add position tracking system for accurate trade history

- Add trader_positions table to track complete open/close trades
- Add PositionSyncManager to detect manual closes via polling
- Record position on open, update on close with PnL calculation
- Use positions table for trading stats and recent trades (replacing orders table)
- Fix TradingView chart symbol format (add .P suffix for futures)
- Fix DecisionCard wait/hold action color (gray instead of red)
- Auto-append USDT suffix for custom symbol input

* update

---------
2025-12-06 01:04:26 +08:00
Rick
7eebb4e218 Dev backtest (#1134) 2025-11-28 21:34:27 +08:00
Lawrence Liu
0188ffb778 fix(database): prevent data loss on Docker restart with WAL mode and graceful shutdown (#817)
* fix(database): prevent data loss on Docker restart with WAL mode and graceful shutdown

Fixes #816

## Problem
Exchange API keys and private keys were being lost after `docker compose restart`.
This P0 bug posed critical security and operational risks.

### Root Cause
1. **SQLite journal_mode=delete**: Traditional rollback journal doesn't protect
   against data loss during non-graceful shutdowns
2. **Incomplete graceful shutdown**: Application relied on `defer database.Close()`
   which may not execute before process termination
3. **Docker grace period**: Default 10s may not be sufficient for cleanup

### Data Loss Scenario
```
User updates exchange config → Backend writes to SQLite → Data in buffer (not fsynced)
→ Docker restart (SIGTERM) → App exits → SQLite never flushes → Data lost
```

## Solution

### 1. Enable WAL Mode (Primary Fix)
- **Before**: `journal_mode=delete` (rollback journal)
- **After**: `journal_mode=WAL` (Write-Ahead Logging)

**Benefits:**
-  Crash-safe even during power loss
-  Better concurrent write performance
-  Atomic commits with durability guarantees

### 2. Improve Graceful Shutdown
**Before:**
```go
<-sigChan
traderManager.StopAll()
// defer database.Close() may not execute in time
```

**After:**
```go
<-sigChan
traderManager.StopAll()    // Step 1: Stop traders
server.Shutdown()          // Step 2: Stop HTTP server (new)
database.Close()           // Step 3: Explicit database close (new)
```

### 3. Increase Docker Grace Period
```yaml
stop_grace_period: 30s  # Allow 30s for graceful shutdown
```

## Changes

### config/database.go
- Enable `PRAGMA journal_mode=WAL` on database initialization
- Set `PRAGMA synchronous=FULL` for data durability
- Add log message confirming WAL mode activation

### api/server.go
- Add `httpServer *http.Server` field to Server struct
- Implement `Shutdown()` method with 5s timeout
- Replace `router.Run()` with `httpServer.ListenAndServe()` for graceful shutdown support
- Add `context` import for shutdown context

### main.go
- Add explicit shutdown sequence:
  1. Stop all traders
  2. Shutdown HTTP server (new)
  3. Close database connection (new)
- Add detailed logging for each shutdown step

### docker-compose.yml
- Add `stop_grace_period: 30s` to backend service

### config/database_test.go (TDD)
- `TestWALModeEnabled`: Verify WAL mode is active
- `TestSynchronousMode`: Verify synchronous=FULL setting
- `TestDataPersistenceAcrossReopen`: Simulate Docker restart scenario
- `TestConcurrentWritesWithWAL`: Verify concurrent write handling

## Test Results

```bash
$ go test -v ./config
=== RUN   TestWALModeEnabled
--- PASS: TestWALModeEnabled (0.25s)
=== RUN   TestSynchronousMode
--- PASS: TestSynchronousMode (0.06s)
=== RUN   TestDataPersistenceAcrossReopen
--- PASS: TestDataPersistenceAcrossReopen (0.05s)
=== RUN   TestConcurrentWritesWithWAL
--- PASS: TestConcurrentWritesWithWAL (0.09s)
PASS
```

All 16 tests pass (including 9 existing + 4 new WAL tests + 3 concurrent tests).

## Impact

**Before:**
- 🔴 Exchange credentials lost on restart
- 🔴 Trading operations disrupted
- 🔴 Security risk from credential re-entry

**After:**
-  Data persistence guaranteed
-  No credential loss after restart
-  Safe graceful shutdown in all scenarios
-  Better concurrent performance

## Acceptance Criteria

- [x] WAL mode enabled in database initialization
- [x] Graceful shutdown explicitly closes database
- [x] Unit tests verify data persistence across restarts
- [x] Docker grace period increased to 30s
- [x] All tests pass

## Deployment Notes

After deploying this fix:
1. Rebuild Docker image: `./start.sh start --build`
2. Existing `config.db` will be automatically converted to WAL mode
3. WAL files (`config.db-wal`, `config.db-shm`) will be created
4. No manual intervention required

## References

- SQLite WAL Mode: https://www.sqlite.org/wal.html
- Go http.Server Graceful Shutdown: https://pkg.go.dev/net/http#Server.Shutdown

* Add config.db* to gitignore
2025-11-09 16:23:00 +08:00
Deloz
3c081d85db fix: 支持 NOFX_BACKEND_PORT 环境变量配置端口 (#764)
- 优先级:环境变量 > 数据库配置 > 默认值 8080
- 修复 .env 中端口配置不生效的问题
- 添加端口来源日志输出
2025-11-08 23:21:36 -05:00
Icyoung
57af6c00a3 Dev Crypto (#730)
* feat: remove admin mode

* feat: bugfix

* feat(crypto): 添加RSA-OAEP + AES-GCM混合加密服务

- 实现CryptoService加密服务,支持RSA-OAEP-2048 + AES-256-GCM混合加密
- 集成数据库层加密,自动加密存储敏感字段(API密钥、私钥等)
- 支持环境变量DATA_ENCRYPTION_KEY配置数据加密密钥
- 适配SQLite数据库加密存储(从PostgreSQL移植)
- 保持Hyperliquid代理钱包处理兼容性
- 更新.gitignore以正确处理crypto模块代码

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: tinkle-community <tinklefund@gmail.com>

* feat(scripts): 添加加密环境一键设置脚本

- setup_encryption.sh: 一键生成RSA密钥对+数据加密密钥+JWT密钥
- generate_rsa_keys.sh: 专业的RSA-2048密钥对生成工具
- generate_data_key.sh: 生成AES-256数据加密密钥和JWT认证密钥
- ENCRYPTION_README.md: 详细的加密系统说明文档
- 支持自动检测现有密钥并只生成缺失的密钥
- 完善的权限管理和安全验证
- 兼容macOS和Linux的跨平台支持

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: tinkle-community <tinklefund@gmail.com>

* feat(api): 添加加密API端点和Gin框架集成

- 新增CryptoHandler处理加密相关API请求
- 提供/api/crypto/public-key端点获取RSA公钥
- 提供/api/crypto/decrypt端点解密敏感数据
- 适配Gin框架的HTTP处理器格式
- 集成CryptoService到API服务器
- 支持前端加密数据传输和解密

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: tinkle-community <tinklefund@gmail.com>

* feat(web): 添加前端加密服务和两阶段密钥输入组件

- CryptoService: Web Crypto API集成,支持RSA-OAEP加密
- TwoStageKeyModal: 安全的两阶段私钥输入组件,支持剪贴板混淆
- 完善国际化翻译支持加密相关UI文本
- 修复TypeScript类型错误和编译问题
- 支持前端敏感数据加密传输到后端
- 增强用户隐私保护和数据安全

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: tinkle-community <tinklefund@gmail.com>

* feat(auth): 增强JWT认证安全性

- 优先使用环境变量JWT_SECRET而不是数据库配置
- 支持通过.env文件安全配置JWT认证密钥
- 保留数据库配置作为回退机制
- 改进JWT密钥来源日志显示
- 增强系统启动时的安全配置检查
- 支持运行时动态JWT密钥切换

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: tinkle-community <tinklefund@gmail.com>

* feat(docker): 集成加密环境变量到Docker部署

- 添加DATA_ENCRYPTION_KEY环境变量传递到容器
- 添加JWT_SECRET环境变量支持
- 挂载secrets目录使容器可访问RSA密钥文件
- 确保容器内加密服务正常工作
- 解决容器启动失败和加密初始化问题
- 完善Docker Compose加密环境配置

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: tinkle-community <tinklefund@gmail.com>

* feat(start): 集成自动加密环境检测和设置

- 增强check_encryption()函数检测JWT_SECRET和DATA_ENCRYPTION_KEY
- 自动运行setup_encryption.sh当检测到缺失密钥时
- 改进加密状态显示,包含RSA+AES+JWT全套加密信息
- 优化用户体验,提供清晰的加密配置反馈
- 支持一键设置完整加密环境
- 确保容器启动前加密环境就绪

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: tinkle-community <tinklefund@gmail.com>

* feat: format fix

* fix(security): 修复前端模型和交易所配置敏感数据明文传输

- 在handleSaveModelConfig中对API密钥进行RSA-OAEP加密
- 在handleSaveExchangeConfig中对API密钥、Secret密钥和Aster私钥进行加密
- 只有非空敏感数据才进行加密处理
- 添加加密失败错误处理和用户友好提示
- 增加encryptionFailed翻译键的中英文支持
- 使用用户ID和会话ID作为加密上下文增强安全性

这修复了之前敏感数据在网络传输中以明文形式发送的安全漏洞。

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: tinkle-community <tinklefund@gmail.com>

* fix(crypto): 修复后端加密服务集成和缺失的加密端点

- 添加Server结构体缺少的cryptoService字段
- 实现handleUpdateModelConfigsEncrypted处理器用于模型配置加密传输
- 修复handleUpdateExchangeConfigsEncrypted中的函数调用
- 在前端API中添加updateModelConfigsEncrypted方法
- 统一RSA密钥路径从secrets/rsa_key改为keys/rsa_private.key
- 确保前端可以使用加密端点安全传输敏感数据
- 兼容原有加密通信模式和二段输入私钥功能

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: tinkle-community <tinklefund@gmail.com>

---------

Co-authored-by: icy <icyoung520@gmail.com>
Co-authored-by: tinkle-community <tinklefund@gmail.com>
2025-11-08 02:03:09 +08:00
Icyoung
f270bc3fee Dev remove admin mode (#723)
* feat: remove admin mode

* feat: bugfix

---------

Co-authored-by: icy <icyoung520@gmail.com>
2025-11-07 23:37:23 +08:00
Shui
179e5c69bf fix(bootstrap module): add bootstrap module to meet future function (#674)
* fix(bootstrap module): add bootstrap module to meet future function

* Fix readme

* Fix panic because log.logger is nil

* fix import

---------

Co-authored-by: zbhan <zbhan@freewheel.tv>
2025-11-07 10:53:10 +08:00
SkywalkerJi
b99a54ef41 fix: Fixed redundant key input fields and corrected formatting on the frontend. (#566)
* Eliminate redundant key input fields in the front-end.

* go / react Formatting.
2025-11-06 01:16:04 +08:00
Burt
5f98e09920 Feat: Enable admin password in admin mode (#540)
* WIP: save local changes before merging

* Enable admin password in admin mode #374
2025-11-05 21:48:28 +08:00
tangmengqiu
61d45ebb0e log: add logrus log lib and add telegram notification push as an option 2025-11-04 23:48:32 -05:00
SkywalkerJi
8a877fe5dc Fixed go fmt code formatting issues. 2025-11-05 01:42:36 +08:00
icy
78b3ce69da Merge branch 'dev' into beta
# Conflicts:
#	config/database.go
#	main.go
2025-11-02 22:11:07 +08:00
yuanshi2016
5e0706bee3 修改Kline获取方式为Websocket缓存。 2025-11-02 17:59:19 +08:00
yuanshi2016
a3d6113dd1 K线获取方式改为websocket组合流. 带重拨机制
流程为下:
1. 启动时使用所有交易员设置的币种(去重) 如果交易员未配置,则使用系统默认
2. 在决策获取K线时 如果没有缓存 则先实时获取后再添加订阅. ps: 适用于Api方式的币种列表
2025-11-02 14:03:13 +08:00
icy
e444cbbbc8 Add beta mode 2025-11-02 02:31:19 +08:00
yuanshi2016
1d77cb7b5d 新增内置AI评分
修改market/data.go Get函数获取K线为流式获取(可以解决传入币种比较多的情况下耗时问题)
market目录下新增文件
main.go 新增运行入口
通过inside_coins=true控制
该评分默认初始化大约需要2分钟左右(因为币种列表比较多,api有限速)
使用时应该注意engine.go下的流动性过滤问题
2025-11-01 15:58:54 +08:00
icy
bf83e89b33 Merge branch 'dev' of github.com:Icyoung/nofx
# Conflicts:
#	manager/trader_manager.go
#	web/src/components/AITradersPage.tsx
2025-11-01 02:23:05 +08:00
icy
ccd4938857 竞赛fix、交易员新增参数 2025-11-01 02:17:11 +08:00
SkywalkerJi
cdd41fb3b1 Fix coin settings 2025-10-31 21:05:53 +08:00
icy
a3f2706ec8 Sync config.json to config.db for manual modify admin_mode and other configs 2025-10-31 15:24:18 +08:00
icy
d3e7b7dbb1 account system、custom prompt 2025-10-31 03:42:01 +08:00
icy
a2cded592b Merge remote tracking branch into local development
- Resolved conflicts in README.md: Combined web-based config updates with multi-exchange support
- Resolved conflicts in main.go: Fixed database initialization and default coin settings
- Resolved conflicts in manager/trader_manager.go: Updated trader management for new database structure
- Resolved conflicts in web/src/App.tsx: Combined UI improvements with responsive design
- Resolved conflicts in web/.dockerignore: Merged dependency exclusions
- Removed deprecated files: Dockerfile, config/config.go, web/Dockerfile, ComparisonChart.tsx, CompetitionPage.tsx

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: tinkle-community <tinklefund@gmail.com>
2025-10-30 20:57:57 +08:00
icy
c64727ffd0 sync fork 2025-10-30 20:51:22 +08:00
SkywalkerJi
7f7ad141a4 Expose the default mainstream currency settings to the configuration file 2025-10-30 10:01:45 +08:00
PorunC
78b091cca8 Feat: Integrate leverage configuration across trading system
- Pass leverage config through TraderManager to AutoTrader
- Add BTCETHLeverage and AltcoinLeverage fields to Context and AutoTraderConfig
- Update decision validation to use configured leverage limits
- Display configured leverage in startup message
- Update error messages to show current leverage limits

Changes:
- main.go: Pass leverage config to AddTrader, update startup message
- manager/trader_manager.go: Accept and forward leverage config
- trader/auto_trader.go: Store leverage config, pass to Context
- decision/engine.go: Use dynamic leverage limits in validation

This completes the leverage configuration feature implementation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: tinkle-community <tinklefund@gmail.com>
2025-10-29 20:30:04 +08:00
nobody
41e3654c11 feat: 添加use_default_coins开关,简化默认币种配置
- 新增use_default_coins配置项:true=使用默认8个主流币种,false=使用API币种池
- 更直观的配置方式,新手可直接设置true快速开始
- 优先级:use_default_coins > coin_pool_api_url > 缓存 > 默认列表
- 更新所有语言版本README(en/zh-CN/ru/uk)添加配置说明
- 更新config.json.example示例文件
2025-10-29 12:59:12 +08:00
tinkle-community
8d3981fd3e Initial commit: NOFX AI Trading System
- Multi-AI competition mode (Qwen vs DeepSeek)
- Binance Futures integration
- AI self-learning mechanism
- Professional web dashboard
- Complete risk management system
2025-10-28 15:47:34 +08:00