Files
nofx/main.go
Lawrence Liu 0188ffb778 fix(database): prevent data loss on Docker restart with WAL mode and graceful shutdown (#817)
* fix(database): prevent data loss on Docker restart with WAL mode and graceful shutdown

Fixes #816

## Problem
Exchange API keys and private keys were being lost after `docker compose restart`.
This P0 bug posed critical security and operational risks.

### Root Cause
1. **SQLite journal_mode=delete**: Traditional rollback journal doesn't protect
   against data loss during non-graceful shutdowns
2. **Incomplete graceful shutdown**: Application relied on `defer database.Close()`
   which may not execute before process termination
3. **Docker grace period**: Default 10s may not be sufficient for cleanup

### Data Loss Scenario
```
User updates exchange config → Backend writes to SQLite → Data in buffer (not fsynced)
→ Docker restart (SIGTERM) → App exits → SQLite never flushes → Data lost
```

## Solution

### 1. Enable WAL Mode (Primary Fix)
- **Before**: `journal_mode=delete` (rollback journal)
- **After**: `journal_mode=WAL` (Write-Ahead Logging)

**Benefits:**
-  Crash-safe even during power loss
-  Better concurrent write performance
-  Atomic commits with durability guarantees

### 2. Improve Graceful Shutdown
**Before:**
```go
<-sigChan
traderManager.StopAll()
// defer database.Close() may not execute in time
```

**After:**
```go
<-sigChan
traderManager.StopAll()    // Step 1: Stop traders
server.Shutdown()          // Step 2: Stop HTTP server (new)
database.Close()           // Step 3: Explicit database close (new)
```

### 3. Increase Docker Grace Period
```yaml
stop_grace_period: 30s  # Allow 30s for graceful shutdown
```

## Changes

### config/database.go
- Enable `PRAGMA journal_mode=WAL` on database initialization
- Set `PRAGMA synchronous=FULL` for data durability
- Add log message confirming WAL mode activation

### api/server.go
- Add `httpServer *http.Server` field to Server struct
- Implement `Shutdown()` method with 5s timeout
- Replace `router.Run()` with `httpServer.ListenAndServe()` for graceful shutdown support
- Add `context` import for shutdown context

### main.go
- Add explicit shutdown sequence:
  1. Stop all traders
  2. Shutdown HTTP server (new)
  3. Close database connection (new)
- Add detailed logging for each shutdown step

### docker-compose.yml
- Add `stop_grace_period: 30s` to backend service

### config/database_test.go (TDD)
- `TestWALModeEnabled`: Verify WAL mode is active
- `TestSynchronousMode`: Verify synchronous=FULL setting
- `TestDataPersistenceAcrossReopen`: Simulate Docker restart scenario
- `TestConcurrentWritesWithWAL`: Verify concurrent write handling

## Test Results

```bash
$ go test -v ./config
=== RUN   TestWALModeEnabled
--- PASS: TestWALModeEnabled (0.25s)
=== RUN   TestSynchronousMode
--- PASS: TestSynchronousMode (0.06s)
=== RUN   TestDataPersistenceAcrossReopen
--- PASS: TestDataPersistenceAcrossReopen (0.05s)
=== RUN   TestConcurrentWritesWithWAL
--- PASS: TestConcurrentWritesWithWAL (0.09s)
PASS
```

All 16 tests pass (including 9 existing + 4 new WAL tests + 3 concurrent tests).

## Impact

**Before:**
- 🔴 Exchange credentials lost on restart
- 🔴 Trading operations disrupted
- 🔴 Security risk from credential re-entry

**After:**
-  Data persistence guaranteed
-  No credential loss after restart
-  Safe graceful shutdown in all scenarios
-  Better concurrent performance

## Acceptance Criteria

- [x] WAL mode enabled in database initialization
- [x] Graceful shutdown explicitly closes database
- [x] Unit tests verify data persistence across restarts
- [x] Docker grace period increased to 30s
- [x] All tests pass

## Deployment Notes

After deploying this fix:
1. Rebuild Docker image: `./start.sh start --build`
2. Existing `config.db` will be automatically converted to WAL mode
3. WAL files (`config.db-wal`, `config.db-shm`) will be created
4. No manual intervention required

## References

- SQLite WAL Mode: https://www.sqlite.org/wal.html
- Go http.Server Graceful Shutdown: https://pkg.go.dev/net/http#Server.Shutdown

* Add config.db* to gitignore
2025-11-09 16:23:00 +08:00

388 lines
13 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
package main
import (
"encoding/json"
"fmt"
"log"
"nofx/api"
"nofx/auth"
"nofx/config"
"nofx/crypto"
"nofx/manager"
"nofx/market"
"nofx/pool"
"os"
"os/signal"
"strconv"
"strings"
"syscall"
"github.com/joho/godotenv"
)
// ConfigFile 配置文件结构,只包含需要同步到数据库的字段
// TODO 现在与config.Config相同未来会被替换 现在为了兼容性不得不保留当前文件
type ConfigFile struct {
BetaMode bool `json:"beta_mode"`
APIServerPort int `json:"api_server_port"`
UseDefaultCoins bool `json:"use_default_coins"`
DefaultCoins []string `json:"default_coins"`
CoinPoolAPIURL string `json:"coin_pool_api_url"`
OITopAPIURL string `json:"oi_top_api_url"`
MaxDailyLoss float64 `json:"max_daily_loss"`
MaxDrawdown float64 `json:"max_drawdown"`
StopTradingMinutes int `json:"stop_trading_minutes"`
Leverage config.LeverageConfig `json:"leverage"`
JWTSecret string `json:"jwt_secret"`
DataKLineTime string `json:"data_k_line_time"`
Log *config.LogConfig `json:"log"` // 日志配置
}
// loadConfigFile 读取并解析config.json文件
func loadConfigFile() (*ConfigFile, error) {
// 检查config.json是否存在
if _, err := os.Stat("config.json"); os.IsNotExist(err) {
log.Printf("📄 config.json不存在使用默认配置")
return &ConfigFile{}, nil
}
// 读取config.json
data, err := os.ReadFile("config.json")
if err != nil {
return nil, fmt.Errorf("读取config.json失败: %w", err)
}
// 解析JSON
var configFile ConfigFile
if err := json.Unmarshal(data, &configFile); err != nil {
return nil, fmt.Errorf("解析config.json失败: %w", err)
}
return &configFile, nil
}
// syncConfigToDatabase 将配置同步到数据库
func syncConfigToDatabase(database *config.Database, configFile *ConfigFile) error {
if configFile == nil {
return nil
}
log.Printf("🔄 开始同步config.json到数据库...")
// 同步各配置项到数据库
configs := map[string]string{
"beta_mode": fmt.Sprintf("%t", configFile.BetaMode),
"api_server_port": strconv.Itoa(configFile.APIServerPort),
"use_default_coins": fmt.Sprintf("%t", configFile.UseDefaultCoins),
"coin_pool_api_url": configFile.CoinPoolAPIURL,
"oi_top_api_url": configFile.OITopAPIURL,
"max_daily_loss": fmt.Sprintf("%.1f", configFile.MaxDailyLoss),
"max_drawdown": fmt.Sprintf("%.1f", configFile.MaxDrawdown),
"stop_trading_minutes": strconv.Itoa(configFile.StopTradingMinutes),
}
// 同步default_coins转换为JSON字符串存储
if len(configFile.DefaultCoins) > 0 {
defaultCoinsJSON, err := json.Marshal(configFile.DefaultCoins)
if err == nil {
configs["default_coins"] = string(defaultCoinsJSON)
}
}
// 同步杠杆配置
if configFile.Leverage.BTCETHLeverage > 0 {
configs["btc_eth_leverage"] = strconv.Itoa(configFile.Leverage.BTCETHLeverage)
}
if configFile.Leverage.AltcoinLeverage > 0 {
configs["altcoin_leverage"] = strconv.Itoa(configFile.Leverage.AltcoinLeverage)
}
// 如果JWT密钥不为空也同步
if configFile.JWTSecret != "" {
configs["jwt_secret"] = configFile.JWTSecret
}
// 更新数据库配置
for key, value := range configs {
if err := database.SetSystemConfig(key, value); err != nil {
log.Printf("⚠️ 更新配置 %s 失败: %v", key, err)
} else {
log.Printf("✓ 同步配置: %s = %s", key, value)
}
}
log.Printf("✅ config.json同步完成")
return nil
}
// loadBetaCodesToDatabase 加载内测码文件到数据库
func loadBetaCodesToDatabase(database *config.Database) error {
betaCodeFile := "beta_codes.txt"
// 检查内测码文件是否存在
if _, err := os.Stat(betaCodeFile); os.IsNotExist(err) {
log.Printf("📄 内测码文件 %s 不存在,跳过加载", betaCodeFile)
return nil
}
// 获取文件信息
fileInfo, err := os.Stat(betaCodeFile)
if err != nil {
return fmt.Errorf("获取内测码文件信息失败: %w", err)
}
log.Printf("🔄 发现内测码文件 %s (%.1f KB),开始加载...", betaCodeFile, float64(fileInfo.Size())/1024)
// 加载内测码到数据库
err = database.LoadBetaCodesFromFile(betaCodeFile)
if err != nil {
return fmt.Errorf("加载内测码失败: %w", err)
}
// 显示统计信息
total, used, err := database.GetBetaCodeStats()
if err != nil {
log.Printf("⚠️ 获取内测码统计失败: %v", err)
} else {
log.Printf("✅ 内测码加载完成: 总计 %d 个,已使用 %d 个,剩余 %d 个", total, used, total-used)
}
return nil
}
func main() {
fmt.Println("╔════════════════════════════════════════════════════════════╗")
fmt.Println("║ 🤖 AI多模型交易系统 - 支持 DeepSeek & Qwen ║")
fmt.Println("╚════════════════════════════════════════════════════════════╝")
fmt.Println()
// Load environment variables from .env file if present (for local/dev runs)
// In Docker Compose, variables are injected by the runtime and this is harmless.
_ = godotenv.Load()
// 初始化数据库配置
dbPath := "config.db"
if len(os.Args) > 1 {
dbPath = os.Args[1]
}
// 读取配置文件
configFile, err := loadConfigFile()
if err != nil {
log.Fatalf("❌ 读取config.json失败: %v", err)
}
log.Printf("📋 初始化配置数据库: %s", dbPath)
database, err := config.NewDatabase(dbPath)
if err != nil {
log.Fatalf("❌ 初始化数据库失败: %v", err)
}
defer database.Close()
// 初始化加密服务
log.Printf("🔐 初始化加密服务...")
cryptoService, err := crypto.NewCryptoService("secrets/rsa_key")
if err != nil {
log.Fatalf("❌ 初始化加密服务失败: %v", err)
}
database.SetCryptoService(cryptoService)
log.Printf("✅ 加密服务初始化成功")
// 同步config.json到数据库
if err := syncConfigToDatabase(database, configFile); err != nil {
log.Printf("⚠️ 同步config.json到数据库失败: %v", err)
}
// 加载内测码到数据库
if err := loadBetaCodesToDatabase(database); err != nil {
log.Printf("⚠️ 加载内测码到数据库失败: %v", err)
}
// 获取系统配置
useDefaultCoinsStr, _ := database.GetSystemConfig("use_default_coins")
useDefaultCoins := useDefaultCoinsStr == "true"
apiPortStr, _ := database.GetSystemConfig("api_server_port")
// 设置JWT密钥优先使用环境变量
jwtSecret := strings.TrimSpace(os.Getenv("JWT_SECRET"))
if jwtSecret == "" {
// 回退到数据库配置
jwtSecret, _ = database.GetSystemConfig("jwt_secret")
if jwtSecret == "" {
jwtSecret = "your-jwt-secret-key-change-in-production-make-it-long-and-random"
log.Printf("⚠️ 使用默认JWT密钥建议使用加密设置脚本生成安全密钥")
} else {
log.Printf("🔑 使用数据库中JWT密钥")
}
} else {
log.Printf("🔑 使用环境变量JWT密钥")
}
auth.SetJWTSecret(jwtSecret)
// 管理员模式下需要管理员密码,缺失则退出
log.Printf("✓ 配置数据库初始化成功")
fmt.Println()
// 从数据库读取默认主流币种列表
defaultCoinsJSON, _ := database.GetSystemConfig("default_coins")
var defaultCoins []string
if defaultCoinsJSON != "" {
// 尝试从JSON解析
if err := json.Unmarshal([]byte(defaultCoinsJSON), &defaultCoins); err != nil {
log.Printf("⚠️ 解析default_coins配置失败: %v使用硬编码默认值", err)
defaultCoins = []string{"BTCUSDT", "ETHUSDT", "SOLUSDT", "BNBUSDT", "XRPUSDT", "DOGEUSDT", "ADAUSDT", "HYPEUSDT"}
} else {
log.Printf("✓ 从数据库加载默认币种列表(共%d个: %v", len(defaultCoins), defaultCoins)
}
} else {
// 如果数据库中没有配置,使用硬编码默认值
defaultCoins = []string{"BTCUSDT", "ETHUSDT", "SOLUSDT", "BNBUSDT", "XRPUSDT", "DOGEUSDT", "ADAUSDT", "HYPEUSDT"}
log.Printf("⚠️ 数据库中未配置default_coins使用硬编码默认值")
}
pool.SetDefaultCoins(defaultCoins)
// 设置是否使用默认主流币种
pool.SetUseDefaultCoins(useDefaultCoins)
if useDefaultCoins {
log.Printf("✓ 已启用默认主流币种列表")
}
// 设置币种池API URL
coinPoolAPIURL, _ := database.GetSystemConfig("coin_pool_api_url")
if coinPoolAPIURL != "" {
pool.SetCoinPoolAPI(coinPoolAPIURL)
log.Printf("✓ 已配置AI500币种池API")
}
oiTopAPIURL, _ := database.GetSystemConfig("oi_top_api_url")
if oiTopAPIURL != "" {
pool.SetOITopAPI(oiTopAPIURL)
log.Printf("✓ 已配置OI Top API")
}
// 创建TraderManager
traderManager := manager.NewTraderManager()
// 从数据库加载所有交易员到内存
err = traderManager.LoadTradersFromDatabase(database)
if err != nil {
log.Fatalf("❌ 加载交易员失败: %v", err)
}
// 获取数据库中的所有交易员配置用于显示使用default用户
traders, err := database.GetTraders("default")
if err != nil {
log.Fatalf("❌ 获取交易员列表失败: %v", err)
}
// 显示加载的交易员信息
fmt.Println()
fmt.Println("🤖 数据库中的AI交易员配置:")
if len(traders) == 0 {
fmt.Println(" • 暂无配置的交易员请通过Web界面创建")
} else {
for _, trader := range traders {
status := "停止"
if trader.IsRunning {
status = "运行中"
}
fmt.Printf(" • %s (%s + %s) - 初始资金: %.0f USDT [%s]\n",
trader.Name, strings.ToUpper(trader.AIModelID), strings.ToUpper(trader.ExchangeID),
trader.InitialBalance, status)
}
}
// 创建初始化上下文
// TODO : 传入实际配置, 现在并未实际使用,未来所有模块初始化都将通过上下文传递配置
// ctx := bootstrap.NewContext(&config.Config{})
// // 执行所有初始化钩子
// if err := bootstrap.Run(ctx); err != nil {
// log.Fatalf("初始化失败: %v", err)
// }
fmt.Println()
fmt.Println("🤖 AI全权决策模式:")
fmt.Printf(" • AI将自主决定每笔交易的杠杆倍数山寨币最高5倍BTC/ETH最高5倍\n")
fmt.Println(" • AI将自主决定每笔交易的仓位大小")
fmt.Println(" • AI将自主设置止损和止盈价格")
fmt.Println(" • AI将基于市场数据、技术指标、账户状态做出全面分析")
fmt.Println()
fmt.Println("⚠️ 风险提示: AI自动交易有风险建议小额资金测试")
fmt.Println()
fmt.Println("按 Ctrl+C 停止运行")
fmt.Println(strings.Repeat("=", 60))
fmt.Println()
// 获取API服务器端口优先级环境变量 > 数据库配置 > 默认值)
apiPort := 8080 // 默认端口
// 1. 优先从环境变量 NOFX_BACKEND_PORT 读取
if envPort := strings.TrimSpace(os.Getenv("NOFX_BACKEND_PORT")); envPort != "" {
if port, err := strconv.Atoi(envPort); err == nil && port > 0 {
apiPort = port
log.Printf("🔌 使用环境变量端口: %d (NOFX_BACKEND_PORT)", apiPort)
} else {
log.Printf("⚠️ 环境变量 NOFX_BACKEND_PORT 无效: %s", envPort)
}
} else if apiPortStr != "" {
// 2. 从数据库配置读取config.json 同步过来的)
if port, err := strconv.Atoi(apiPortStr); err == nil && port > 0 {
apiPort = port
log.Printf("🔌 使用数据库配置端口: %d (api_server_port)", apiPort)
}
} else {
log.Printf("🔌 使用默认端口: %d", apiPort)
}
// 创建并启动API服务器
apiServer := api.NewServer(traderManager, database, cryptoService, apiPort)
go func() {
if err := apiServer.Start(); err != nil {
log.Printf("❌ API服务器错误: %v", err)
}
}()
// 启动流行情数据 - 默认使用所有交易员设置的币种 如果没有设置币种 则优先使用系统默认
go market.NewWSMonitor(150).Start(database.GetCustomCoins())
//go market.NewWSMonitor(150).Start([]string{}) //这里是一个使用方式 传入空的话 则使用market市场的所有币种
// 设置优雅退出
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
// TODO: 启动数据库中配置为运行状态的交易员
// traderManager.StartAll()
// 等待退出信号
<-sigChan
fmt.Println()
fmt.Println()
log.Println("📛 收到退出信号,正在优雅关闭...")
// 步骤 1: 停止所有交易员
log.Println("⏸️ 停止所有交易员...")
traderManager.StopAll()
log.Println("✅ 所有交易员已停止")
// 步骤 2: 关闭 API 服务器
log.Println("🛑 停止 API 服务器...")
if err := apiServer.Shutdown(); err != nil {
log.Printf("⚠️ 关闭 API 服务器时出错: %v", err)
} else {
log.Println("✅ API 服务器已安全关闭")
}
// 步骤 3: 关闭数据库连接 (确保所有写入完成)
log.Println("💾 关闭数据库连接...")
if err := database.Close(); err != nil {
log.Printf("❌ 关闭数据库失败: %v", err)
} else {
log.Println("✅ 数据库已安全关闭,所有数据已持久化")
}
fmt.Println()
fmt.Println("👋 感谢使用AI交易系统")
}