{"id":"6a0d2237-99c5-46e4-9c64-0942874ee4e1","authorId":"08398a34-26f5-4992-82de-0cfba0302908","title":"Autopilot Strategy Creation & Evolution with AI Agents and Deep RL","slug":"autopilot-strategy-creation-evolution-with-ai-agents-and-deep-rl","excerpt":"A guide on using the RLXBT Deep Reinforcement Learning (DQN) engine and MCP integrations to train, evaluate, and predict market signals dynamically using AI agents.","content":"# Autopilot Strategy Creation & Evolution with AI Agents and Deep RL\n\n*Category: Tutorials & Guides*  \n*Target Readers: Quantitative Researchers, Algo Traders, Developers*  \n\n---\n\n## Introduction\nOne of the most powerful features of the **RLXBT** ecosystem is the seamless integration of **Deep Reinforcement Learning (Deep RL)** and **AI Agents** via the Model Context Protocol (MCP). Instead of hardcoding static trading rules, quantitative researchers can deploy deep Q-learning networks (`DQN`) that dynamically learn profitable trading policies directly from market indicators.\n\nThis guide details how the RL engine operates under the hood, how the LLM agent interacts with it, and how you can train, evaluate, and query models for real-time predictions.\n\n---\n\n## 1. The Architecture: Two-Level Optimization\nThe system operates on two distinct optimization layers:\n\n```\n+-------------------------------------------------------+\n|                 1. Strategic Layer (LLM Agent)        |\n|  - Defines State space (features) & reward formulas   |\n|  - Triggers experiments via MCP tools                 |\n|  - Evolves hypotheses on the Canvas                   |\n+-------------------------------------------------------+\n                           |\n                           v  (MCP JSON-RPC)\n+-------------------------------------------------------+\n|                 2. Execution Layer (Rust Core)        |\n|  - Compiles RLEnvironment and calculates features     |\n|  - Trains Deep Q-Network (LibTorch) on past data      |\n|  - Runs millisecond-level inference (rl_predict)       |\n+-------------------------------------------------------+\n```\n\n1. **Strategic Layer (AI Agent)**: The LLM structures the research, modifies the features, sets boundaries, and reviews evaluation metrics (Sharpe, WFE, Monte Carlo).\n2. **Execution Layer (Rust Daemon)**: A lightning-fast engine written in optimized Rust that trains neural networks and runs real-time inference in milliseconds.\n\n---\n\n## 2. Core MCP Tools for Reinforcement Learning\nThe `rlxbt` MCP server exposes three main endpoints to manage the Deep RL lifecycle:\n\n### A. Training: `rl_train`\nLaunches a full reinforcement learning training cycle. The environment compiles custom indicator matrices, normalizes the state vectors, and trains the DQN weights using PyTorch.\n* **Key Arguments**:\n  * `episodes`: Number of training passes (e.g., 10 to 50).\n  * `window_size`: Lookback window (e.g., 20 bars).\n  * `reward_type`: Optimization target (`portfolio`, `log`, `sharpe`, `calmar`, or `risk`).\n  * `train_split`: Training/validation boundary (e.g., 0.8).\n* **Return**: Report ID containing the serialized neural weights.\n\n### B. Evaluation: `rl_evaluate`\nRuns the trained model over an entire unseen dataset (Out-of-Sample) to calculate standard trading metrics without training noise.\n* **Key Arguments**:\n  * `id`: The report ID of the trained model.\n  * `features_path`: Path to the evaluation dataset.\n* **Return**: Sharpe ratio, total return, max drawdown, win rate, action distribution (flat/long/short), and equity curve.\n\n### C. Live Inference: `rl_predict`\nInterrogates the model for an immediate signal by passing raw OHLCV bars. The Rust environment dynamically compiles indicators on the fly.\n* **Key Arguments**:\n  * `id`: Report ID.\n  * `bars`: An array of recent bars (at least `window_size + 1` length).\n* **Return**: Recommended action (`long`, `short`, `flat`), direction (`1`, `-1`, `0`), confidence percentage, and Q-values.\n\n---\n\n## 3. Example: Fetching Live Signals in Python\nHere is how to query your trained model for real-time predictions using the running daemon bridge:\n\n```python\nimport requests\nimport json\n\n# Setup the bridge url and report ID\nBRIDGE_URL = \"http://127.0.0.1:8145\"\nREPORT_ID = \"rpt_1783598697206_18\"  # Replace with your model's ID\n\n# Recent OHLCV bars (must be >= window_size + 1)\nrecent_bars = [\n    {\"timestamp\": 1783519200, \"open\": 61945.49, \"high\": 62173.56, \"low\": 61712.0, \"close\": 61889.83, \"volume\": 1415.74},\n    # ... Add at least 20 more historical bars ...\n]\n\n# JSON-RPC request format\npayload = {\n    \"jsonrpc\": \"2.0\",\n    \"id\": 1,\n    \"method\": \"tools/call\",\n    \"params\": {\n        \"name\": \"rl_predict\",\n        \"arguments\": {\n            \"id\": REPORT_ID,\n            \"bars\": recent_bars\n        }\n    }\n}\n\nresponse = requests.post(BRIDGE_URL, json=payload)\nresult = json.loads(response.json()[\"result\"][\"content\"][0][\"text\"])\n\nif result[\"status\"] == \"success\":\n    signal = result[\"data\"]\n    print(f\"Action: {signal['action'].upper()} (Confidence: {signal['confidence']*100:.2f}%)\")\nelse:\n    print(\"Error:\", result[\"message\"])\n```\n\n---\n\n## 4. Evolving Strategies on the Canvas\n1. **Define a Hypothesis**: Set up your indicators (e.g., `drift_z_24` and `absorption_24`) as observations on the Idea Map.\n2. **Train the RL Agent**: Call `rl_train` to let the DQN discover trading patterns on those features.\n3. **Validate**: Call `rl_evaluate` on a separate out-of-sample fold to calculate the Walk-Forward Efficiency.\n4. **Deploy**: Feed real-time exchange WebSocket feeds directly into `rl_predict` to execute live actions.\n","coverImage":null,"status":"published","publishedAt":"2026-07-09T12:07:38.479Z","backtestResults":{"bars":10000,"asset":"BTCUSDT","metrics":{"sharpe":2.338,"trades":150,"win_rate":0.5067,"max_drawdown":0.0585,"total_return":0.1572},"verdict":"robust","strategy":{"exit_rules":[],"entry_rules":[{"signal":"DQN_RL_Policy","condition":"DQN RL model inference","direction":1}]},"timeframe":"1h","robustness":{"sensitivity_top_param":"window_size","walk_forward_efficiency":0,"monte_carlo_risk_of_ruin":0},"tools_used":["rl_train","rl_evaluate","rl_predict"]},"researchManifest":null,"viewCount":124,"likeCount":0,"metaTitle":null,"metaDescription":null,"createdAt":"2026-07-09T12:07:38.481Z","updatedAt":"2026-09-11T11:29:58.912Z","author":{"id":"08398a34-26f5-4992-82de-0cfba0302908","name":"Serg","picture":"https://lh3.googleusercontent.com/a/ACg8ocKJfy0qxMGacsuTCbRKqF2-Ahj7AqOXwzIJU2wBJye6JmKsh8A=s96-c"},"tags":[{"id":"36df6628-764a-45f1-9a15-3d8c92933836","name":"rl","slug":"rl","color":"#6366f1","createdAt":"2026-07-09T12:07:38.492Z"},{"id":"1f67cc2f-bf74-44a3-b782-4590dbf77dcd","name":"dqn","slug":"dqn","color":"#6366f1","createdAt":"2026-07-09T12:07:38.498Z"},{"id":"9b2bf8b5-325f-4fc3-843b-18070872eadf","name":"mcp","slug":"mcp","color":"#6366f1","createdAt":"2026-07-09T12:07:38.503Z"},{"id":"548ebc89-f20d-4aa6-ba6f-464a23c7a978","name":"agent","slug":"agent","color":"#6366f1","createdAt":"2026-07-02T14:44:23.006Z"},{"id":"ffadd99f-87cb-4f7a-b9cf-81e921f6309d","name":"prediction","slug":"prediction","color":"#6366f1","createdAt":"2026-07-09T12:07:38.510Z"},{"id":"95b56f8d-c0c3-499e-a2f1-3ba3da291b5e","name":"tutorial","slug":"tutorial","color":"#6366f1","createdAt":"2026-07-09T12:07:38.514Z"}],"comments":[],"isLiked":false,"isAuthor":false}